Jun 09 01:15:38.865817 ip-10-0-143-161 systemd[1]: Starting Kubernetes Kubelet... Jun 09 01:15:39.257715 ip-10-0-143-161 kubenswrapper[2555]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 01:15:39.257715 ip-10-0-143-161 kubenswrapper[2555]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 09 01:15:39.257715 ip-10-0-143-161 kubenswrapper[2555]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 01:15:39.257715 ip-10-0-143-161 kubenswrapper[2555]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 09 01:15:39.257715 ip-10-0-143-161 kubenswrapper[2555]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 01:15:39.258327 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.257790 2555 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 09 01:15:39.261698 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261680 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 01:15:39.261698 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261698 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261706 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261711 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261716 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261720 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261724 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261734 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261739 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261743 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261747 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261751 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261754 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261759 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261762 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261766 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261770 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261774 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261778 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261782 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 01:15:39.261843 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261787 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261791 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261794 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261799 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261803 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261807 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261814 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261819 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261824 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261828 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261832 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261837 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261840 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261845 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261849 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261854 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261858 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261862 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261866 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261870 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 01:15:39.262427 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261875 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261879 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261884 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261888 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261892 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261896 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261900 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261904 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261917 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261922 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261926 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261930 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261934 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261938 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261943 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261947 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261952 2555 feature_gate.go:328] unrecognized feature gate: Example Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261957 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261961 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261965 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 01:15:39.262935 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261970 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261974 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261978 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261982 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261986 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261993 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.261999 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262004 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262009 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262013 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262018 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262022 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262029 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262034 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262039 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262043 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262048 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262052 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262056 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 01:15:39.263469 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262060 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262064 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262068 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262072 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262077 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262080 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262085 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262091 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262095 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262099 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262103 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262107 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262112 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262116 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262121 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262125 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262129 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262134 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262162 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262179 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 01:15:39.263952 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262946 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262954 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262958 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262962 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262965 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262968 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262971 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262974 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262977 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262980 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262982 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262985 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262987 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262990 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262994 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.262998 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263001 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263004 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263007 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 01:15:39.264437 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263010 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263013 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263016 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263019 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263022 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263024 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263027 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263030 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263033 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263035 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263038 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263040 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263044 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263047 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263050 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263053 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263055 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263058 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263060 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263063 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 01:15:39.264941 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263065 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263068 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263071 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263074 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263076 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263079 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263081 2555 feature_gate.go:328] unrecognized feature gate: Example Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263085 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263089 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263092 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263095 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263098 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263101 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263103 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263106 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263108 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263111 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263113 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263116 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 01:15:39.265438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263118 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263121 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263123 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263129 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263131 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263134 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263139 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263141 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263144 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263146 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263149 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263151 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263154 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263156 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263159 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263161 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263164 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263166 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263169 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263171 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 01:15:39.265928 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263173 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263176 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263178 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263181 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263185 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263188 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263191 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263193 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263196 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263198 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263200 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263203 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263205 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263208 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263211 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263214 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263217 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263220 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263223 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 01:15:39.266423 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263226 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263229 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263597 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263602 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263606 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263609 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263612 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263614 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263617 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263619 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263622 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263625 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263627 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263630 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263633 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263636 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263638 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263641 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263644 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263647 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 01:15:39.266900 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263649 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263652 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263655 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263657 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263659 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263662 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263664 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263667 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263671 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263674 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263676 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263679 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263682 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263686 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263690 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263693 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263696 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263699 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263702 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 01:15:39.267382 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263704 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263707 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263710 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263713 2555 feature_gate.go:328] unrecognized feature gate: Example Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263716 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263718 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263721 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263725 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263728 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263730 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263733 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263736 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263739 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263741 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263743 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263746 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263748 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263751 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263753 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263756 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 01:15:39.267857 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263758 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263762 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263765 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263768 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263770 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263773 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263776 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263778 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263781 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263784 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263786 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263789 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263792 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263795 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263798 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263801 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263804 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263806 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263809 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263812 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 01:15:39.268404 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263814 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263817 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263819 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263822 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263824 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263827 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263830 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263833 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263835 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263838 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263840 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263843 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263845 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263850 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263854 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263857 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263859 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263862 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263865 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263868 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 01:15:39.268916 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263871 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.263874 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.263940 2555 flags.go:64] FLAG: --address="0.0.0.0" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.263947 2555 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.263953 2555 flags.go:64] FLAG: --anonymous-auth="true" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.263958 2555 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.263962 2555 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.263966 2555 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.263970 2555 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.263974 2555 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.263977 2555 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.263980 2555 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.263983 2555 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.263986 2555 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.263990 2555 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.263993 2555 flags.go:64] FLAG: --cgroup-root="" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.263996 2555 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.263999 2555 flags.go:64] FLAG: --client-ca-file="" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264002 2555 flags.go:64] FLAG: --cloud-provider="external" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264004 2555 flags.go:64] FLAG: --cluster-dns="[]" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264008 2555 flags.go:64] FLAG: --cluster-domain="" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264011 2555 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264014 2555 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264017 2555 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 09 01:15:39.269399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264020 2555 flags.go:64] FLAG: --container-log-max-files="5" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264025 2555 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264028 2555 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264032 2555 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264035 2555 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264038 2555 flags.go:64] FLAG: --contention-profiling="false" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264040 2555 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264043 2555 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264048 2555 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264051 2555 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264055 2555 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264058 2555 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264061 2555 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264064 2555 flags.go:64] FLAG: --enable-load-reader="false" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264067 2555 flags.go:64] FLAG: --enable-server="true" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264069 2555 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264073 2555 flags.go:64] FLAG: --event-burst="100" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264076 2555 flags.go:64] FLAG: --event-qps="50" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264079 2555 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264082 2555 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264085 2555 flags.go:64] FLAG: --eviction-hard="" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264088 2555 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264091 2555 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264094 2555 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264097 2555 flags.go:64] FLAG: --eviction-soft="" Jun 09 01:15:39.269988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264100 2555 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264103 2555 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264105 2555 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264108 2555 flags.go:64] FLAG: --experimental-mounter-path="" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264111 2555 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264114 2555 flags.go:64] FLAG: --fail-swap-on="true" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264117 2555 flags.go:64] FLAG: --feature-gates="" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264120 2555 flags.go:64] FLAG: --file-check-frequency="20s" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264124 2555 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264127 2555 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264130 2555 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264134 2555 flags.go:64] FLAG: --healthz-port="10248" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264137 2555 flags.go:64] FLAG: --help="false" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264140 2555 flags.go:64] FLAG: --hostname-override="ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264143 2555 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264147 2555 flags.go:64] FLAG: --http-check-frequency="20s" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264151 2555 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264154 2555 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264157 2555 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264160 2555 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264163 2555 flags.go:64] FLAG: --image-service-endpoint="" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264166 2555 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264169 2555 flags.go:64] FLAG: --kube-api-burst="100" Jun 09 01:15:39.270603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264171 2555 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264174 2555 flags.go:64] FLAG: --kube-api-qps="50" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264177 2555 flags.go:64] FLAG: --kube-reserved="" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264180 2555 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264183 2555 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264186 2555 flags.go:64] FLAG: --kubelet-cgroups="" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264188 2555 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264191 2555 flags.go:64] FLAG: --lock-file="" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264194 2555 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264197 2555 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264200 2555 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264206 2555 flags.go:64] FLAG: --log-json-split-stream="false" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264209 2555 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264212 2555 flags.go:64] FLAG: --log-text-split-stream="false" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264214 2555 flags.go:64] FLAG: --logging-format="text" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264217 2555 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264220 2555 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264224 2555 flags.go:64] FLAG: --manifest-url="" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264227 2555 flags.go:64] FLAG: --manifest-url-header="" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264231 2555 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264234 2555 flags.go:64] FLAG: --max-open-files="1000000" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264238 2555 flags.go:64] FLAG: --max-pods="110" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264241 2555 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264244 2555 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264248 2555 flags.go:64] FLAG: --memory-manager-policy="None" Jun 09 01:15:39.271308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264251 2555 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264262 2555 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264265 2555 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264269 2555 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264280 2555 flags.go:64] FLAG: --node-status-max-images="50" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264283 2555 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264285 2555 flags.go:64] FLAG: --oom-score-adj="-999" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264289 2555 flags.go:64] FLAG: --pod-cidr="" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264292 2555 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264297 2555 flags.go:64] FLAG: --pod-manifest-path="" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264300 2555 flags.go:64] FLAG: --pod-max-pids="-1" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264303 2555 flags.go:64] FLAG: --pods-per-core="0" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264306 2555 flags.go:64] FLAG: --port="10250" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264309 2555 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264311 2555 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0433993f7a3d7066e" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264314 2555 flags.go:64] FLAG: --qos-reserved="" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264317 2555 flags.go:64] FLAG: --read-only-port="10255" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264320 2555 flags.go:64] FLAG: --register-node="true" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264323 2555 flags.go:64] FLAG: --register-with-taints="" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264327 2555 flags.go:64] FLAG: --registry-burst="10" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264329 2555 flags.go:64] FLAG: --registry-qps="5" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264332 2555 flags.go:64] FLAG: --reserved-cpus="" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264335 2555 flags.go:64] FLAG: --reserved-memory="" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264339 2555 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264342 2555 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 09 01:15:39.272231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264346 2555 flags.go:64] FLAG: --rotate-certificates="false" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264349 2555 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264351 2555 flags.go:64] FLAG: --runonce="false" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264354 2555 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264357 2555 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264360 2555 flags.go:64] FLAG: --seccomp-default="false" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264364 2555 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264367 2555 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264370 2555 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264374 2555 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264376 2555 flags.go:64] FLAG: --storage-driver-password="root" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264379 2555 flags.go:64] FLAG: --storage-driver-secure="false" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264382 2555 flags.go:64] FLAG: --storage-driver-table="stats" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264385 2555 flags.go:64] FLAG: --storage-driver-user="root" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264388 2555 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264391 2555 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264394 2555 flags.go:64] FLAG: --system-cgroups="" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264397 2555 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264402 2555 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264404 2555 flags.go:64] FLAG: --tls-cert-file="" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264407 2555 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264411 2555 flags.go:64] FLAG: --tls-min-version="" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264414 2555 flags.go:64] FLAG: --tls-private-key-file="" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264416 2555 flags.go:64] FLAG: --topology-manager-policy="none" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264419 2555 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 09 01:15:39.272933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264422 2555 flags.go:64] FLAG: --topology-manager-scope="container" Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264425 2555 flags.go:64] FLAG: --v="2" Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264428 2555 flags.go:64] FLAG: --version="false" Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264432 2555 flags.go:64] FLAG: --vmodule="" Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264436 2555 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.264439 2555 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264589 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264597 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264600 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264603 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264606 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264608 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264611 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264616 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264618 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264622 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264624 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264628 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264631 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264633 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 01:15:39.273622 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264636 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264639 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264641 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264644 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264646 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264649 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264652 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264654 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264657 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264659 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264662 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264664 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264667 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264669 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264672 2555 feature_gate.go:328] unrecognized feature gate: Example Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264674 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264677 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264679 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264682 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264686 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 01:15:39.274167 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264688 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264691 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264693 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264697 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264701 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264705 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264708 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264710 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264713 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264715 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264720 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264723 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264726 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264728 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264731 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264734 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264736 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264739 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264742 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 01:15:39.274729 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264744 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264747 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264749 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264752 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264754 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264757 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264759 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264762 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264764 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264767 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264770 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264772 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264776 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264779 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264790 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264794 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264797 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264801 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264805 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264808 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 01:15:39.275237 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264810 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264813 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264815 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264818 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264822 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264825 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264829 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264832 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264835 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264838 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264841 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264843 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264846 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264849 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264852 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264854 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264857 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264859 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264861 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264864 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 01:15:39.275797 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264867 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 01:15:39.276372 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264869 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 01:15:39.276372 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264872 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 01:15:39.276372 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264874 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 01:15:39.276372 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264879 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 01:15:39.276372 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.264881 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 01:15:39.276372 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.265487 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 01:15:39.276372 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.273153 2555 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 09 01:15:39.276372 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.273170 2555 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 09 01:15:39.276372 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.273189 2555 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 09 01:15:39.276372 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.273195 2555 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 09 01:15:39.276372 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273252 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 01:15:39.276372 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273257 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 01:15:39.276372 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273260 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 01:15:39.276372 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273263 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273266 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273268 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273271 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273274 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273277 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273279 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273283 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273285 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273288 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273291 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273293 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273296 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273299 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273303 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273307 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273310 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273313 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273315 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273318 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 01:15:39.276741 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273321 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273323 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273326 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273329 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273332 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273334 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273338 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273340 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273343 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273346 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273349 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273351 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273354 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273357 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273359 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273361 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273364 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273367 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273370 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273373 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 01:15:39.277233 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273375 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273378 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273380 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273384 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273387 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273389 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273392 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273394 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273397 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273399 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273402 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273404 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273407 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273409 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273412 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273414 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273417 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273419 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273423 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273425 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 01:15:39.277794 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273429 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273431 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273434 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273438 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273441 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273444 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273446 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273449 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273452 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273454 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273457 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273459 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273462 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273464 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273467 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273469 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273472 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273475 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273477 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 01:15:39.278263 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273480 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 01:15:39.278743 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273483 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 01:15:39.278743 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273486 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 01:15:39.278743 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273488 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 01:15:39.278743 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273491 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 01:15:39.278743 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273495 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 01:15:39.278743 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273498 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 01:15:39.278743 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273501 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 01:15:39.278743 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273504 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 01:15:39.278743 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273507 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 01:15:39.278743 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273510 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 01:15:39.278743 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273513 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 01:15:39.278743 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273516 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 01:15:39.278743 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273519 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 01:15:39.278743 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273522 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 01:15:39.278743 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273524 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 01:15:39.278743 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273527 2555 feature_gate.go:328] unrecognized feature gate: Example Jun 09 01:15:39.279150 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.273532 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 01:15:39.279150 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273664 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 01:15:39.279150 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273694 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 01:15:39.279150 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273698 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 01:15:39.279150 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273701 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 01:15:39.279150 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273704 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 01:15:39.279150 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273707 2555 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 01:15:39.279150 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273710 2555 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 01:15:39.279150 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273713 2555 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 01:15:39.279150 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273716 2555 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 01:15:39.279150 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273720 2555 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 01:15:39.279150 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273724 2555 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 01:15:39.279150 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273727 2555 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 01:15:39.279150 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273731 2555 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273734 2555 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273736 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273739 2555 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273742 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273745 2555 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273748 2555 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273751 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273754 2555 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273757 2555 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273760 2555 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273762 2555 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273765 2555 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273768 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273771 2555 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273774 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273777 2555 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273779 2555 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273782 2555 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 01:15:39.279501 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273785 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273788 2555 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273791 2555 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273795 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273798 2555 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273801 2555 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273803 2555 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273805 2555 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273808 2555 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273811 2555 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273813 2555 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273816 2555 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273818 2555 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273821 2555 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273824 2555 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273826 2555 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273829 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273831 2555 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273834 2555 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273836 2555 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 01:15:39.279980 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273839 2555 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273842 2555 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273844 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273846 2555 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273849 2555 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273851 2555 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273854 2555 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273857 2555 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273860 2555 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273862 2555 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273865 2555 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273867 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273870 2555 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273872 2555 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273874 2555 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273877 2555 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273880 2555 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273882 2555 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273885 2555 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273887 2555 feature_gate.go:328] unrecognized feature gate: Example Jun 09 01:15:39.280462 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273889 2555 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273892 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273894 2555 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273897 2555 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273899 2555 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273902 2555 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273904 2555 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273909 2555 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273911 2555 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273914 2555 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273916 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273919 2555 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273921 2555 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273924 2555 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273926 2555 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273929 2555 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273932 2555 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273934 2555 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273937 2555 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273940 2555 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 01:15:39.280967 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273943 2555 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 01:15:39.281470 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273946 2555 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 01:15:39.281470 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273948 2555 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 01:15:39.281470 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273951 2555 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 01:15:39.281470 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273953 2555 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 01:15:39.281470 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273956 2555 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 01:15:39.281470 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273958 2555 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 01:15:39.281470 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:39.273961 2555 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 01:15:39.281470 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.273965 2555 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 01:15:39.281470 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.274060 2555 server.go:962] "Client rotation is on, will bootstrap in background" Jun 09 01:15:39.281470 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.277504 2555 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 09 01:15:39.281470 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.278880 2555 server.go:1019] "Starting client certificate rotation" Jun 09 01:15:39.281470 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.278978 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 09 01:15:39.281470 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.279018 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 09 01:15:39.281838 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.281538 2555 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 09 01:15:39.285573 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.285556 2555 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 09 01:15:39.298403 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.298389 2555 log.go:25] "Validated CRI v1 runtime API" Jun 09 01:15:39.302056 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.302040 2555 log.go:25] "Validated CRI v1 image API" Jun 09 01:15:39.303339 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.303319 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 09 01:15:39.303647 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.303632 2555 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 09 01:15:39.307623 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.307606 2555 fs.go:135] Filesystem UUIDs: map[173353d7-0aeb-456f-92f1-67862486f9ee:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9a9d88f1-541c-493e-a563-f0e15246de0a:/dev/nvme0n1p4] Jun 09 01:15:39.307680 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.307623 2555 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 09 01:15:39.315448 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.315333 2555 manager.go:216] Machine: {Timestamp:2026-06-09 01:15:39.313575638 +0000 UTC m=+0.331879846 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099628 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20d4f1202f0ef5563b39bf70b8ffd2 SystemUUID:ec20d4f1-202f-0ef5-563b-39bf70b8ffd2 BootID:74b612df-3c5e-43cd-be04-d926c3fbf2fe Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fe:1a:7b:62:e7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fe:1a:7b:62:e7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:a2:12:4e:b5:69:00 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 09 01:15:39.315448 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.315436 2555 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 09 01:15:39.315600 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.315501 2555 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 09 01:15:39.319056 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.319026 2555 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 09 01:15:39.319178 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.319059 2555 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-161.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 09 01:15:39.319817 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.319808 2555 topology_manager.go:138] "Creating topology manager with none policy" Jun 09 01:15:39.319848 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.319820 2555 container_manager_linux.go:309] "Creating device plugin manager" Jun 09 01:15:39.319848 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.319828 2555 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 09 01:15:39.320555 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.320532 2555 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 09 01:15:39.320941 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.320927 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p47x4" Jun 09 01:15:39.321471 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.321460 2555 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 09 01:15:39.324809 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.324798 2555 state_mem.go:36] "Initialized new in-memory state store" Jun 09 01:15:39.324917 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.324908 2555 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 09 01:15:39.324950 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.324937 2555 kubelet.go:486] "Attempting to sync node with API server" Jun 09 01:15:39.324950 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.324945 2555 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 09 01:15:39.325006 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.324959 2555 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 09 01:15:39.325006 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.324968 2555 kubelet.go:398] "Adding apiserver pod source" Jun 09 01:15:39.325006 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.324976 2555 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 09 01:15:39.325239 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.325228 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 09 01:15:39.325281 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.325245 2555 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 09 01:15:39.327700 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.327686 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p47x4" Jun 09 01:15:39.327899 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.327883 2555 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 09 01:15:39.329905 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.329891 2555 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 09 01:15:39.329947 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.329921 2555 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 09 01:15:39.331124 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.331109 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 09 01:15:39.331168 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.331133 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 09 01:15:39.331168 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.331142 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 09 01:15:39.331168 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.331151 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 09 01:15:39.331168 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.331163 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 09 01:15:39.331168 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.331170 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 09 01:15:39.331302 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.331180 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 09 01:15:39.331302 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.331187 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 09 01:15:39.331302 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.331197 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 09 01:15:39.331302 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.331202 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 09 01:15:39.331302 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.331211 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 09 01:15:39.331302 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.331216 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 09 01:15:39.332635 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.332620 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 09 01:15:39.332684 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.332638 2555 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 09 01:15:39.335748 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.335736 2555 server.go:1268] "Started kubelet" Jun 09 01:15:39.335884 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.335826 2555 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 09 01:15:39.335977 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.335914 2555 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 09 01:15:39.336043 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.335990 2555 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 09 01:15:39.336702 ip-10-0-143-161 systemd[1]: Started Kubernetes Kubelet. Jun 09 01:15:39.337648 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.337498 2555 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 01:15:39.338730 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.338715 2555 server.go:310] "Adding debug handlers to kubelet server" Jun 09 01:15:39.340402 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.340322 2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 01:15:39.341922 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.340880 2555 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 09 01:15:39.342729 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:39.342679 2555 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 09 01:15:39.342809 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.342740 2555 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-161.ec2.internal" not found Jun 09 01:15:39.345418 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.345400 2555 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 09 01:15:39.345418 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.345412 2555 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 09 01:15:39.345570 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:39.345491 2555 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-161.ec2.internal\" not found" Jun 09 01:15:39.346191 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.346168 2555 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 09 01:15:39.346279 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.346176 2555 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 09 01:15:39.346279 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.346207 2555 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 09 01:15:39.346368 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.346293 2555 reconstruct.go:97] "Volume reconstruction finished" Jun 09 01:15:39.346368 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.346303 2555 reconciler.go:29] "Reconciler: start to sync state" Jun 09 01:15:39.346368 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.346363 2555 factory.go:55] Registering systemd factory Jun 09 01:15:39.346470 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.346374 2555 factory.go:223] Registration of the systemd container factory successfully Jun 09 01:15:39.346596 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.346584 2555 factory.go:153] Registering CRI-O factory Jun 09 01:15:39.346648 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.346598 2555 factory.go:223] Registration of the crio container factory successfully Jun 09 01:15:39.346648 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.346638 2555 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 09 01:15:39.346718 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.346660 2555 factory.go:103] Registering Raw factory Jun 09 01:15:39.346718 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.346673 2555 manager.go:1196] Started watching for new ooms in manager Jun 09 01:15:39.347106 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.347092 2555 manager.go:349] Starting recovery of all containers Jun 09 01:15:39.347649 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.347627 2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 01:15:39.350638 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:39.350612 2555 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-161.ec2.internal\" not found" node="ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.356422 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.356408 2555 manager.go:354] Recovery completed Jun 09 01:15:39.358129 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.358113 2555 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-161.ec2.internal" not found Jun 09 01:15:39.361721 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.361709 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 01:15:39.363363 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.363347 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientMemory" Jun 09 01:15:39.363433 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.363378 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 01:15:39.363433 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.363388 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientPID" Jun 09 01:15:39.364872 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.364857 2555 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 09 01:15:39.364872 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.364871 2555 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 09 01:15:39.365024 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.364889 2555 state_mem.go:36] "Initialized new in-memory state store" Jun 09 01:15:39.365306 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.364957 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 01:15:39.366098 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.366083 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientMemory" Jun 09 01:15:39.366169 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.366110 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 01:15:39.366169 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.366121 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientPID" Jun 09 01:15:39.366292 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.366281 2555 policy_none.go:49] "None policy: Start" Jun 09 01:15:39.366330 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.366296 2555 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 09 01:15:39.366330 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.366306 2555 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 09 01:15:39.367367 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.367356 2555 policy_none.go:47] "Start" Jun 09 01:15:39.400141 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.400120 2555 manager.go:337] "Starting Device Plugin manager" Jun 09 01:15:39.411016 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:39.400147 2555 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 09 01:15:39.411016 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.400155 2555 server.go:87] "Starting device plugin registration server" Jun 09 01:15:39.411016 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.400422 2555 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 09 01:15:39.411016 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.400432 2555 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 09 01:15:39.411016 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.400659 2555 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 09 01:15:39.411016 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.400747 2555 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 09 01:15:39.411016 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.400753 2555 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 09 01:15:39.411016 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:39.401087 2555 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 09 01:15:39.411016 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:39.401134 2555 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-161.ec2.internal\" not found" Jun 09 01:15:39.411016 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.407897 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 09 01:15:39.411016 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.409064 2555 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 09 01:15:39.411016 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.409121 2555 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 09 01:15:39.411016 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.409143 2555 kubelet.go:2448] "Starting kubelet main sync loop" Jun 09 01:15:39.411016 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:39.409180 2555 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 09 01:15:39.411758 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.411295 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 01:15:39.421042 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.421028 2555 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-143-161.ec2.internal" not found Jun 09 01:15:39.501422 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.501404 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 01:15:39.503790 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.503770 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientMemory" Jun 09 01:15:39.503877 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.503800 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 01:15:39.503877 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.503811 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientPID" Jun 09 01:15:39.503877 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.503830 2555 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.510033 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.509326 2555 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal"] Jun 09 01:15:39.510100 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.510034 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 01:15:39.510741 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.510726 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientMemory" Jun 09 01:15:39.510805 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.510752 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 01:15:39.510805 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.510762 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientPID" Jun 09 01:15:39.511983 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.511971 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 01:15:39.512113 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.512099 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.512147 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.512132 2555 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 01:15:39.512624 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.512608 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientMemory" Jun 09 01:15:39.512678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.512608 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientMemory" Jun 09 01:15:39.512678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.512638 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 01:15:39.512678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.512649 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientPID" Jun 09 01:15:39.512678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.512657 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 01:15:39.512678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.512669 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientPID" Jun 09 01:15:39.513471 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.513457 2555 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.513993 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.513976 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.541071 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.541053 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.545456 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.545443 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.545561 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.545533 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.547369 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.547353 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a29d25d72d9ebc3da303cb0225e4c752-config\") pod \"kube-apiserver-proxy-ip-10-0-143-161.ec2.internal\" (UID: \"a29d25d72d9ebc3da303cb0225e4c752\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.547451 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.547380 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2018821b48cefdd51ae3152836bd5175-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal\" (UID: \"2018821b48cefdd51ae3152836bd5175\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.547451 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.547406 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2018821b48cefdd51ae3152836bd5175-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal\" (UID: \"2018821b48cefdd51ae3152836bd5175\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.554861 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.554844 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 09 01:15:39.554954 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.554862 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 09 01:15:39.563384 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.563366 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 09 01:15:39.563466 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:39.563406 2555 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.563466 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.563415 2555 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.578441 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.578424 2555 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 09 01:15:39.578509 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:39.578453 2555 kubelet.go:3250] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-143-161.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.647782 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.647764 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2018821b48cefdd51ae3152836bd5175-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal\" (UID: \"2018821b48cefdd51ae3152836bd5175\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.647867 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.647833 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2018821b48cefdd51ae3152836bd5175-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal\" (UID: \"2018821b48cefdd51ae3152836bd5175\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.647911 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.647845 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2018821b48cefdd51ae3152836bd5175-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal\" (UID: \"2018821b48cefdd51ae3152836bd5175\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.647911 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.647901 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a29d25d72d9ebc3da303cb0225e4c752-config\") pod \"kube-apiserver-proxy-ip-10-0-143-161.ec2.internal\" (UID: \"a29d25d72d9ebc3da303cb0225e4c752\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.647978 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.647927 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a29d25d72d9ebc3da303cb0225e4c752-config\") pod \"kube-apiserver-proxy-ip-10-0-143-161.ec2.internal\" (UID: \"a29d25d72d9ebc3da303cb0225e4c752\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.647978 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.647862 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2018821b48cefdd51ae3152836bd5175-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal\" (UID: \"2018821b48cefdd51ae3152836bd5175\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.859177 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.859129 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" Jun 09 01:15:39.860267 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:39.860251 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal" Jun 09 01:15:40.278020 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.277966 2555 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 09 01:15:40.278430 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.278105 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 09 01:15:40.278430 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.278141 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 09 01:15:40.278430 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.278162 2555 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 09 01:15:40.325352 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.325331 2555 apiserver.go:52] "Watching apiserver" Jun 09 01:15:40.331755 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.331712 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-08 01:10:39 +0000 UTC" deadline="2028-01-15 13:58:37.850746069 +0000 UTC" Jun 09 01:15:40.331755 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.331751 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14052h42m57.518997567s" Jun 09 01:15:40.333620 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.333603 2555 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 09 01:15:40.334019 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.334000 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-s4jjf","openshift-network-operator/iptables-alerter-2jddj","kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9","openshift-dns/node-resolver-rs95s","openshift-image-registry/node-ca-87mlr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal","openshift-multus/multus-additional-cni-plugins-5zrlt","openshift-multus/network-metrics-daemon-8prh2","openshift-ovn-kubernetes/ovnkube-node-gpkxb","kube-system/global-pull-secret-syncer-wgbm8","kube-system/konnectivity-agent-xlqwl","openshift-cluster-node-tuning-operator/tuned-nfwxf","openshift-multus/multus-sl99s"] Jun 09 01:15:40.335651 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.335627 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:15:40.335731 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:40.335710 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:15:40.337402 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.337380 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2jddj" Jun 09 01:15:40.339141 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.339115 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.340142 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.340120 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 09 01:15:40.340314 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.340298 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 09 01:15:40.340380 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.340330 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 09 01:15:40.340466 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.340447 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gx2h9\"" Jun 09 01:15:40.340920 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.340905 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rs95s" Jun 09 01:15:40.342070 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.342051 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 09 01:15:40.342163 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.342097 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lw9cc\"" Jun 09 01:15:40.342163 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.342104 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 09 01:15:40.342163 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.342105 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 09 01:15:40.343154 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.342826 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-87mlr" Jun 09 01:15:40.343738 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.343716 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 09 01:15:40.343830 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.343742 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 09 01:15:40.344114 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.344096 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-stcvq\"" Jun 09 01:15:40.345100 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.345071 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.345669 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.345642 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 09 01:15:40.345772 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.345702 2555 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 09 01:15:40.346227 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.346207 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 09 01:15:40.346365 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.346345 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 09 01:15:40.346492 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.346472 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-b95r4\"" Jun 09 01:15:40.346569 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.346517 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:40.346704 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:40.346683 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:15:40.348019 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.347995 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mz8kl\"" Jun 09 01:15:40.348091 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.348063 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.348590 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.348571 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 09 01:15:40.348698 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.348677 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 09 01:15:40.348917 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.348901 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 09 01:15:40.348975 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.348938 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 09 01:15:40.349009 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.348983 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 09 01:15:40.349616 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.349599 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:40.349706 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:40.349648 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:15:40.351287 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351264 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8dxj\" (UniqueName: \"kubernetes.io/projected/c85ec566-05db-4e9c-a563-0615d3a76f06-kube-api-access-l8dxj\") pod \"iptables-alerter-2jddj\" (UID: \"c85ec566-05db-4e9c-a563-0615d3a76f06\") " pod="openshift-network-operator/iptables-alerter-2jddj" Jun 09 01:15:40.351379 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351291 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-etc-selinux\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.351379 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351306 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r62gc\" (UniqueName: \"kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc\") pod \"network-check-target-s4jjf\" (UID: \"42e3c300-dc21-44d9-a8a1-6cd16b11dc77\") " pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:15:40.351379 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351345 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-slash\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.351528 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351385 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.351528 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351425 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 09 01:15:40.351528 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351436 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.351528 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351472 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 09 01:15:40.351528 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351473 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4108bdff-ba06-42ab-9b64-a7be1cf9c8a4-host\") pod \"node-ca-87mlr\" (UID: \"4108bdff-ba06-42ab-9b64-a7be1cf9c8a4\") " pod="openshift-image-registry/node-ca-87mlr" Jun 09 01:15:40.351798 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351566 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwwzm\" (UniqueName: \"kubernetes.io/projected/4108bdff-ba06-42ab-9b64-a7be1cf9c8a4-kube-api-access-kwwzm\") pod \"node-ca-87mlr\" (UID: \"4108bdff-ba06-42ab-9b64-a7be1cf9c8a4\") " pod="openshift-image-registry/node-ca-87mlr" Jun 09 01:15:40.351798 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351594 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-run-openvswitch\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.351798 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351605 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 09 01:15:40.351798 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351625 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-node-log\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.351798 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351630 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 09 01:15:40.351798 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351673 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a3a9620d-b3b8-48ae-b007-acd921f2aeac-env-overrides\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.351798 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351712 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.351798 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351738 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67w8w\" (UniqueName: \"kubernetes.io/projected/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-kube-api-access-67w8w\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.351798 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351770 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-systemd-units\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.351798 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351787 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xlqwl" Jun 09 01:15:40.352257 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351797 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-run-systemd\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.352257 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351859 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-registration-dir\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.352257 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351901 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-os-release\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.352257 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.351964 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.352257 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352018 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rqlg\" (UniqueName: \"kubernetes.io/projected/8e1cf77d-cef0-4363-a02f-f0887da4e984-kube-api-access-5rqlg\") pod \"node-resolver-rs95s\" (UID: \"8e1cf77d-cef0-4363-a02f-f0887da4e984\") " pod="openshift-dns/node-resolver-rs95s" Jun 09 01:15:40.352257 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352064 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-var-lib-openvswitch\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.352257 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352087 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-etc-openvswitch\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.352257 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352108 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-log-socket\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.352257 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352119 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dgg5w\"" Jun 09 01:15:40.352257 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352133 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c85ec566-05db-4e9c-a563-0615d3a76f06-iptables-alerter-script\") pod \"iptables-alerter-2jddj\" (UID: \"c85ec566-05db-4e9c-a563-0615d3a76f06\") " pod="openshift-network-operator/iptables-alerter-2jddj" Jun 09 01:15:40.352257 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352188 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 09 01:15:40.352257 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352205 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-device-dir\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.352257 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352234 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dddg\" (UniqueName: \"kubernetes.io/projected/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-kube-api-access-6dddg\") pod \"network-metrics-daemon-8prh2\" (UID: \"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19\") " pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:40.352257 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352260 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-run-netns\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.352949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352288 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-run-ovn\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.352949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352311 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a3a9620d-b3b8-48ae-b007-acd921f2aeac-ovnkube-script-lib\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.352949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352335 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-socket-dir\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.352949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352358 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-cnibin\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.352949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352379 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8e1cf77d-cef0-4363-a02f-f0887da4e984-hosts-file\") pod \"node-resolver-rs95s\" (UID: \"8e1cf77d-cef0-4363-a02f-f0887da4e984\") " pod="openshift-dns/node-resolver-rs95s" Jun 09 01:15:40.352949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352396 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-run-ovn-kubernetes\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.352949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352425 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3a9620d-b3b8-48ae-b007-acd921f2aeac-ovn-node-metrics-cert\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.352949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352449 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-sys-fs\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.352949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352463 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw9qm\" (UniqueName: \"kubernetes.io/projected/744d9eed-638b-4a43-a4d6-e118377f5139-kube-api-access-nw9qm\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.352949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352493 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-system-cni-dir\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.352949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352499 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 09 01:15:40.352949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352511 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-cni-binary-copy\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.352949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352527 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e1cf77d-cef0-4363-a02f-f0887da4e984-tmp-dir\") pod \"node-resolver-rs95s\" (UID: \"8e1cf77d-cef0-4363-a02f-f0887da4e984\") " pod="openshift-dns/node-resolver-rs95s" Jun 09 01:15:40.352949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352565 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4108bdff-ba06-42ab-9b64-a7be1cf9c8a4-serviceca\") pod \"node-ca-87mlr\" (UID: \"4108bdff-ba06-42ab-9b64-a7be1cf9c8a4\") " pod="openshift-image-registry/node-ca-87mlr" Jun 09 01:15:40.352949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352598 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-kubelet\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.352949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352615 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-cni-bin\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.352949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352636 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs\") pod \"network-metrics-daemon-8prh2\" (UID: \"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19\") " pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:40.353705 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352667 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-cni-netd\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.353705 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352689 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.353705 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352704 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a3a9620d-b3b8-48ae-b007-acd921f2aeac-ovnkube-config\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.353705 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352729 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvhk9\" (UniqueName: \"kubernetes.io/projected/a3a9620d-b3b8-48ae-b007-acd921f2aeac-kube-api-access-wvhk9\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.353705 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.352755 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c85ec566-05db-4e9c-a563-0615d3a76f06-host-slash\") pod \"iptables-alerter-2jddj\" (UID: \"c85ec566-05db-4e9c-a563-0615d3a76f06\") " pod="openshift-network-operator/iptables-alerter-2jddj" Jun 09 01:15:40.353705 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.353627 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.354705 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.354684 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 09 01:15:40.354801 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.354754 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 09 01:15:40.354801 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.354774 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wfwj6\"" Jun 09 01:15:40.355297 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.355284 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.356072 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.356057 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 09 01:15:40.356141 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.356077 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 09 01:15:40.356384 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.356372 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-949d7\"" Jun 09 01:15:40.357800 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.357787 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 09 01:15:40.357999 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.357985 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-f9574\"" Jun 09 01:15:40.358887 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.358872 2555 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 09 01:15:40.384840 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.384823 2555 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-995bf" Jun 09 01:15:40.394355 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.394333 2555 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-995bf" Jun 09 01:15:40.395158 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:40.395133 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2018821b48cefdd51ae3152836bd5175.slice/crio-2fabf401a9a45a0f7143e5ec8a5b306f3fde783f3cc6a1fce561c458e4f9476c WatchSource:0}: Error finding container 2fabf401a9a45a0f7143e5ec8a5b306f3fde783f3cc6a1fce561c458e4f9476c: Status 404 returned error can't find the container with id 2fabf401a9a45a0f7143e5ec8a5b306f3fde783f3cc6a1fce561c458e4f9476c Jun 09 01:15:40.395598 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:40.395580 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda29d25d72d9ebc3da303cb0225e4c752.slice/crio-e73d04ad9d66e986d92117cb38140f82bf43178482034c6b8315d996495be5e2 WatchSource:0}: Error finding container e73d04ad9d66e986d92117cb38140f82bf43178482034c6b8315d996495be5e2: Status 404 returned error can't find the container with id e73d04ad9d66e986d92117cb38140f82bf43178482034c6b8315d996495be5e2 Jun 09 01:15:40.400074 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.400060 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 01:15:40.411740 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.411701 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" event={"ID":"2018821b48cefdd51ae3152836bd5175","Type":"ContainerStarted","Data":"2fabf401a9a45a0f7143e5ec8a5b306f3fde783f3cc6a1fce561c458e4f9476c"} Jun 09 01:15:40.412787 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.412757 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal" event={"ID":"a29d25d72d9ebc3da303cb0225e4c752","Type":"ContainerStarted","Data":"e73d04ad9d66e986d92117cb38140f82bf43178482034c6b8315d996495be5e2"} Jun 09 01:15:40.447267 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.447251 2555 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 09 01:15:40.453029 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453013 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-socket-dir\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.453079 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453037 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8e1cf77d-cef0-4363-a02f-f0887da4e984-hosts-file\") pod \"node-resolver-rs95s\" (UID: \"8e1cf77d-cef0-4363-a02f-f0887da4e984\") " pod="openshift-dns/node-resolver-rs95s" Jun 09 01:15:40.453079 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453053 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-run-ovn-kubernetes\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.453079 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453069 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3a9620d-b3b8-48ae-b007-acd921f2aeac-ovn-node-metrics-cert\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.453217 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453083 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-sys-fs\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.453217 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453133 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-sys-fs\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.453217 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453133 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/8e1cf77d-cef0-4363-a02f-f0887da4e984-hosts-file\") pod \"node-resolver-rs95s\" (UID: \"8e1cf77d-cef0-4363-a02f-f0887da4e984\") " pod="openshift-dns/node-resolver-rs95s" Jun 09 01:15:40.453217 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453158 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-run-ovn-kubernetes\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.453217 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453163 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw9qm\" (UniqueName: \"kubernetes.io/projected/744d9eed-638b-4a43-a4d6-e118377f5139-kube-api-access-nw9qm\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.453217 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453182 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-socket-dir\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.453217 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453213 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-system-cni-dir\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.453558 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453248 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e1cf77d-cef0-4363-a02f-f0887da4e984-tmp-dir\") pod \"node-resolver-rs95s\" (UID: \"8e1cf77d-cef0-4363-a02f-f0887da4e984\") " pod="openshift-dns/node-resolver-rs95s" Jun 09 01:15:40.453558 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453268 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4108bdff-ba06-42ab-9b64-a7be1cf9c8a4-serviceca\") pod \"node-ca-87mlr\" (UID: \"4108bdff-ba06-42ab-9b64-a7be1cf9c8a4\") " pod="openshift-image-registry/node-ca-87mlr" Jun 09 01:15:40.453558 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453250 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-system-cni-dir\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.453558 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453283 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-kubelet\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.453558 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453325 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-cni-bin\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.453558 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453328 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-kubelet\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.453558 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453351 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-system-cni-dir\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.453558 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453367 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-cni-bin\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.453558 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453376 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-multus-daemon-config\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.453558 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453399 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-kubernetes\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.453558 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453423 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-cni-netd\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.453558 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453470 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a3a9620d-b3b8-48ae-b007-acd921f2aeac-ovnkube-config\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.453558 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453471 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-cni-netd\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.453558 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453502 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvhk9\" (UniqueName: \"kubernetes.io/projected/a3a9620d-b3b8-48ae-b007-acd921f2aeac-kube-api-access-wvhk9\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.453558 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453525 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e1cf77d-cef0-4363-a02f-f0887da4e984-tmp-dir\") pod \"node-resolver-rs95s\" (UID: \"8e1cf77d-cef0-4363-a02f-f0887da4e984\") " pod="openshift-dns/node-resolver-rs95s" Jun 09 01:15:40.454326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453589 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.454326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453651 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-sys\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.454326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453675 2555 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 09 01:15:40.454326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453716 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4108bdff-ba06-42ab-9b64-a7be1cf9c8a4-serviceca\") pod \"node-ca-87mlr\" (UID: \"4108bdff-ba06-42ab-9b64-a7be1cf9c8a4\") " pod="openshift-image-registry/node-ca-87mlr" Jun 09 01:15:40.454326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453747 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvkqj\" (UniqueName: \"kubernetes.io/projected/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-kube-api-access-jvkqj\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.454326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453778 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-cni-binary-copy\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.454326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453807 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-multus-conf-dir\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.454326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453836 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.454326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453845 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/268a2882-9374-4669-8ed7-fefeee12a63e-agent-certs\") pod \"konnectivity-agent-xlqwl\" (UID: \"268a2882-9374-4669-8ed7-fefeee12a63e\") " pod="kube-system/konnectivity-agent-xlqwl" Jun 09 01:15:40.454326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453899 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-modprobe-d\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.454326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453931 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwwzm\" (UniqueName: \"kubernetes.io/projected/4108bdff-ba06-42ab-9b64-a7be1cf9c8a4-kube-api-access-kwwzm\") pod \"node-ca-87mlr\" (UID: \"4108bdff-ba06-42ab-9b64-a7be1cf9c8a4\") " pod="openshift-image-registry/node-ca-87mlr" Jun 09 01:15:40.454326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.453959 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-node-log\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.454326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454004 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a3a9620d-b3b8-48ae-b007-acd921f2aeac-ovnkube-config\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.454326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454056 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a3a9620d-b3b8-48ae-b007-acd921f2aeac-env-overrides\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.454326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454074 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.454326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454096 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67w8w\" (UniqueName: \"kubernetes.io/projected/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-kube-api-access-67w8w\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.454326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454057 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-node-log\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.455038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454118 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-var-lib-cni-bin\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.455038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454150 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-sysctl-d\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.455038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454169 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.455038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454226 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-var-lib-kubelet\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.455038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454263 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-systemd-units\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.455038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454292 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-run-systemd\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.455038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454318 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-registration-dir\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.455038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454327 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-run-systemd\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.455038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454326 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-systemd-units\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.455038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454344 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-os-release\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.455038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454365 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a3a9620d-b3b8-48ae-b007-acd921f2aeac-env-overrides\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.455038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454373 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/76027187-053a-4879-b929-163968a1cd82-dbus\") pod \"global-pull-secret-syncer-wgbm8\" (UID: \"76027187-053a-4879-b929-163968a1cd82\") " pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:40.455038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454393 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-run\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.455038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454396 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-registration-dir\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.455038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454416 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rqlg\" (UniqueName: \"kubernetes.io/projected/8e1cf77d-cef0-4363-a02f-f0887da4e984-kube-api-access-5rqlg\") pod \"node-resolver-rs95s\" (UID: \"8e1cf77d-cef0-4363-a02f-f0887da4e984\") " pod="openshift-dns/node-resolver-rs95s" Jun 09 01:15:40.455038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454436 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-etc-openvswitch\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.455038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454437 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-os-release\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.455763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454470 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-etc-openvswitch\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.455763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454458 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c85ec566-05db-4e9c-a563-0615d3a76f06-iptables-alerter-script\") pod \"iptables-alerter-2jddj\" (UID: \"c85ec566-05db-4e9c-a563-0615d3a76f06\") " pod="openshift-network-operator/iptables-alerter-2jddj" Jun 09 01:15:40.455763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454508 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-device-dir\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.455763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454535 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-os-release\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.455763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454577 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-var-lib-kubelet\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.455763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454599 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-device-dir\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.455763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454604 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dddg\" (UniqueName: \"kubernetes.io/projected/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-kube-api-access-6dddg\") pod \"network-metrics-daemon-8prh2\" (UID: \"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19\") " pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:40.455763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454643 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-run-netns\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.455763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454712 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-run-ovn\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.455763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454717 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-run-netns\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.455763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454741 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a3a9620d-b3b8-48ae-b007-acd921f2aeac-ovnkube-script-lib\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.455763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454761 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-run-ovn\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.455763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454787 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-cnibin\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.455763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454880 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c85ec566-05db-4e9c-a563-0615d3a76f06-iptables-alerter-script\") pod \"iptables-alerter-2jddj\" (UID: \"c85ec566-05db-4e9c-a563-0615d3a76f06\") " pod="openshift-network-operator/iptables-alerter-2jddj" Jun 09 01:15:40.455763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454888 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-cnibin\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.455763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454890 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-multus-cni-dir\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.455763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454919 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-cni-binary-copy\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.456270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454934 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-cnibin\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.456270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454949 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-run-k8s-cni-cncf-io\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.456270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454967 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-var-lib-cni-multus\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.456270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.454998 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret\") pod \"global-pull-secret-syncer-wgbm8\" (UID: \"76027187-053a-4879-b929-163968a1cd82\") " pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:40.456270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455018 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-sysctl-conf\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.456270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455031 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-lib-modules\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.456270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455046 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-run-netns\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.456270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455067 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-hostroot\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.456270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455106 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75kr7\" (UniqueName: \"kubernetes.io/projected/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-kube-api-access-75kr7\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.456270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455140 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs\") pod \"network-metrics-daemon-8prh2\" (UID: \"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19\") " pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:40.456270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455159 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.456270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455178 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c85ec566-05db-4e9c-a563-0615d3a76f06-host-slash\") pod \"iptables-alerter-2jddj\" (UID: \"c85ec566-05db-4e9c-a563-0615d3a76f06\") " pod="openshift-network-operator/iptables-alerter-2jddj" Jun 09 01:15:40.456270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455196 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8dxj\" (UniqueName: \"kubernetes.io/projected/c85ec566-05db-4e9c-a563-0615d3a76f06-kube-api-access-l8dxj\") pod \"iptables-alerter-2jddj\" (UID: \"c85ec566-05db-4e9c-a563-0615d3a76f06\") " pod="openshift-network-operator/iptables-alerter-2jddj" Jun 09 01:15:40.456270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455228 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.456270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455234 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a3a9620d-b3b8-48ae-b007-acd921f2aeac-ovnkube-script-lib\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.456270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455247 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-etc-selinux\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.456270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455240 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c85ec566-05db-4e9c-a563-0615d3a76f06-host-slash\") pod \"iptables-alerter-2jddj\" (UID: \"c85ec566-05db-4e9c-a563-0615d3a76f06\") " pod="openshift-network-operator/iptables-alerter-2jddj" Jun 09 01:15:40.456744 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455265 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-multus-socket-dir-parent\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.456744 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:40.455293 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 01:15:40.456744 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455339 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/744d9eed-638b-4a43-a4d6-e118377f5139-etc-selinux\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.456744 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455374 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-etc-kubernetes\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.456744 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455398 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-host\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.456744 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:40.455418 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs podName:8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19 nodeName:}" failed. No retries permitted until 2026-06-09 01:15:40.95539925 +0000 UTC m=+1.973703437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs") pod "network-metrics-daemon-8prh2" (UID: "8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 01:15:40.456744 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455464 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-cni-binary-copy\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.456744 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455476 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r62gc\" (UniqueName: \"kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc\") pod \"network-check-target-s4jjf\" (UID: \"42e3c300-dc21-44d9-a8a1-6cd16b11dc77\") " pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:15:40.456744 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455505 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-slash\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.456744 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455530 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.456744 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455593 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-host-slash\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.456744 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455648 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/268a2882-9374-4669-8ed7-fefeee12a63e-konnectivity-ca\") pod \"konnectivity-agent-xlqwl\" (UID: \"268a2882-9374-4669-8ed7-fefeee12a63e\") " pod="kube-system/konnectivity-agent-xlqwl" Jun 09 01:15:40.456744 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455682 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-systemd\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.456744 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455714 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-tuned\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.456744 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455759 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4108bdff-ba06-42ab-9b64-a7be1cf9c8a4-host\") pod \"node-ca-87mlr\" (UID: \"4108bdff-ba06-42ab-9b64-a7be1cf9c8a4\") " pod="openshift-image-registry/node-ca-87mlr" Jun 09 01:15:40.456744 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455804 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4108bdff-ba06-42ab-9b64-a7be1cf9c8a4-host\") pod \"node-ca-87mlr\" (UID: \"4108bdff-ba06-42ab-9b64-a7be1cf9c8a4\") " pod="openshift-image-registry/node-ca-87mlr" Jun 09 01:15:40.456744 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455805 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-run-openvswitch\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.457254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455831 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.457254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455840 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-run-openvswitch\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.457254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455849 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-run-multus-certs\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.457254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455872 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-sysconfig\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.457254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455962 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-var-lib-openvswitch\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.457254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455984 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.457254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.455991 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-var-lib-openvswitch\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.457254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.456014 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-log-socket\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.457254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.456037 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/76027187-053a-4879-b929-163968a1cd82-kubelet-config\") pod \"global-pull-secret-syncer-wgbm8\" (UID: \"76027187-053a-4879-b929-163968a1cd82\") " pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:40.457254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.456052 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-tmp\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.457254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.456085 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a3a9620d-b3b8-48ae-b007-acd921f2aeac-log-socket\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.457254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.456254 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.457631 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.457496 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a3a9620d-b3b8-48ae-b007-acd921f2aeac-ovn-node-metrics-cert\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.461963 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.461861 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw9qm\" (UniqueName: \"kubernetes.io/projected/744d9eed-638b-4a43-a4d6-e118377f5139-kube-api-access-nw9qm\") pod \"aws-ebs-csi-driver-node-kjlt9\" (UID: \"744d9eed-638b-4a43-a4d6-e118377f5139\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.462248 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.462220 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67w8w\" (UniqueName: \"kubernetes.io/projected/fbd5c78f-a01c-41d6-81ce-fd01b8b78e43-kube-api-access-67w8w\") pod \"multus-additional-cni-plugins-5zrlt\" (UID: \"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43\") " pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.462499 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.462480 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwwzm\" (UniqueName: \"kubernetes.io/projected/4108bdff-ba06-42ab-9b64-a7be1cf9c8a4-kube-api-access-kwwzm\") pod \"node-ca-87mlr\" (UID: \"4108bdff-ba06-42ab-9b64-a7be1cf9c8a4\") " pod="openshift-image-registry/node-ca-87mlr" Jun 09 01:15:40.462592 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.462506 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvhk9\" (UniqueName: \"kubernetes.io/projected/a3a9620d-b3b8-48ae-b007-acd921f2aeac-kube-api-access-wvhk9\") pod \"ovnkube-node-gpkxb\" (UID: \"a3a9620d-b3b8-48ae-b007-acd921f2aeac\") " pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.463231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.463215 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dddg\" (UniqueName: \"kubernetes.io/projected/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-kube-api-access-6dddg\") pod \"network-metrics-daemon-8prh2\" (UID: \"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19\") " pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:40.463375 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:40.463359 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 01:15:40.463417 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:40.463384 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 01:15:40.463417 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:40.463398 2555 projected.go:196] Error preparing data for projected volume kube-api-access-r62gc for pod openshift-network-diagnostics/network-check-target-s4jjf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 01:15:40.463488 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.463436 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rqlg\" (UniqueName: \"kubernetes.io/projected/8e1cf77d-cef0-4363-a02f-f0887da4e984-kube-api-access-5rqlg\") pod \"node-resolver-rs95s\" (UID: \"8e1cf77d-cef0-4363-a02f-f0887da4e984\") " pod="openshift-dns/node-resolver-rs95s" Jun 09 01:15:40.463488 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:40.463466 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc podName:42e3c300-dc21-44d9-a8a1-6cd16b11dc77 nodeName:}" failed. No retries permitted until 2026-06-09 01:15:40.963449364 +0000 UTC m=+1.981753553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r62gc" (UniqueName: "kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc") pod "network-check-target-s4jjf" (UID: "42e3c300-dc21-44d9-a8a1-6cd16b11dc77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 01:15:40.465069 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.465050 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8dxj\" (UniqueName: \"kubernetes.io/projected/c85ec566-05db-4e9c-a563-0615d3a76f06-kube-api-access-l8dxj\") pod \"iptables-alerter-2jddj\" (UID: \"c85ec566-05db-4e9c-a563-0615d3a76f06\") " pod="openshift-network-operator/iptables-alerter-2jddj" Jun 09 01:15:40.557099 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557055 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-system-cni-dir\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.557099 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557081 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-multus-daemon-config\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.557099 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557095 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-kubernetes\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.557250 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557113 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-sys\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.557250 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557153 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvkqj\" (UniqueName: \"kubernetes.io/projected/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-kube-api-access-jvkqj\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.557250 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557163 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-sys\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.557250 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557180 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-cni-binary-copy\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.557250 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557195 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-multus-conf-dir\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.557250 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557198 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-kubernetes\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.557250 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557209 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/268a2882-9374-4669-8ed7-fefeee12a63e-agent-certs\") pod \"konnectivity-agent-xlqwl\" (UID: \"268a2882-9374-4669-8ed7-fefeee12a63e\") " pod="kube-system/konnectivity-agent-xlqwl" Jun 09 01:15:40.557250 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557216 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-system-cni-dir\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.557250 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557229 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-modprobe-d\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557257 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-var-lib-cni-bin\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557266 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-multus-conf-dir\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557300 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-sysctl-d\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557316 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-var-lib-cni-bin\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557324 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-var-lib-kubelet\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557354 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-modprobe-d\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557372 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/76027187-053a-4879-b929-163968a1cd82-dbus\") pod \"global-pull-secret-syncer-wgbm8\" (UID: \"76027187-053a-4879-b929-163968a1cd82\") " pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557397 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-run\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557401 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-sysctl-d\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557414 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-var-lib-kubelet\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557421 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-os-release\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557463 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-run\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557474 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-os-release\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557465 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/76027187-053a-4879-b929-163968a1cd82-dbus\") pod \"global-pull-secret-syncer-wgbm8\" (UID: \"76027187-053a-4879-b929-163968a1cd82\") " pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557517 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-var-lib-kubelet\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557575 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-multus-cni-dir\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557602 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-cnibin\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.557678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557601 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-var-lib-kubelet\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557629 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-run-k8s-cni-cncf-io\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557671 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-cnibin\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557680 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-multus-cni-dir\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557653 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-var-lib-cni-multus\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557706 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-multus-daemon-config\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557711 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-run-k8s-cni-cncf-io\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557721 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-var-lib-cni-multus\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557729 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret\") pod \"global-pull-secret-syncer-wgbm8\" (UID: \"76027187-053a-4879-b929-163968a1cd82\") " pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557739 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-cni-binary-copy\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557751 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-sysctl-conf\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:40.557796 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557821 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-sysctl-conf\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:40.557836 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret podName:76027187-053a-4879-b929-163968a1cd82 nodeName:}" failed. No retries permitted until 2026-06-09 01:15:41.057822213 +0000 UTC m=+2.076126395 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret") pod "global-pull-secret-syncer-wgbm8" (UID: "76027187-053a-4879-b929-163968a1cd82") : object "kube-system"/"original-pull-secret" not registered Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557855 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-lib-modules\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557897 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-run-netns\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557922 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-hostroot\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557941 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-lib-modules\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.558252 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557980 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-hostroot\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.557985 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-run-netns\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558059 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75kr7\" (UniqueName: \"kubernetes.io/projected/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-kube-api-access-75kr7\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558153 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-multus-socket-dir-parent\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558188 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-etc-kubernetes\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558210 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-host\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558230 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-etc-kubernetes\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558240 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-multus-socket-dir-parent\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558251 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/268a2882-9374-4669-8ed7-fefeee12a63e-konnectivity-ca\") pod \"konnectivity-agent-xlqwl\" (UID: \"268a2882-9374-4669-8ed7-fefeee12a63e\") " pod="kube-system/konnectivity-agent-xlqwl" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558272 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-host\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558276 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-systemd\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558298 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-tuned\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558324 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-run-multus-certs\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558346 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-sysconfig\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558353 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-systemd\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558400 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-sysconfig\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558399 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-host-run-multus-certs\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558419 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/76027187-053a-4879-b929-163968a1cd82-kubelet-config\") pod \"global-pull-secret-syncer-wgbm8\" (UID: \"76027187-053a-4879-b929-163968a1cd82\") " pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:40.558946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558450 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-tmp\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.559448 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.558468 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/76027187-053a-4879-b929-163968a1cd82-kubelet-config\") pod \"global-pull-secret-syncer-wgbm8\" (UID: \"76027187-053a-4879-b929-163968a1cd82\") " pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:40.559448 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.559409 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/268a2882-9374-4669-8ed7-fefeee12a63e-konnectivity-ca\") pod \"konnectivity-agent-xlqwl\" (UID: \"268a2882-9374-4669-8ed7-fefeee12a63e\") " pod="kube-system/konnectivity-agent-xlqwl" Jun 09 01:15:40.559505 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.559441 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/268a2882-9374-4669-8ed7-fefeee12a63e-agent-certs\") pod \"konnectivity-agent-xlqwl\" (UID: \"268a2882-9374-4669-8ed7-fefeee12a63e\") " pod="kube-system/konnectivity-agent-xlqwl" Jun 09 01:15:40.560151 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.560135 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-etc-tuned\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.561063 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.561049 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-tmp\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.565825 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.565808 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvkqj\" (UniqueName: \"kubernetes.io/projected/b98d9165-4521-4f00-9bf4-3ed9da4f53d8-kube-api-access-jvkqj\") pod \"tuned-nfwxf\" (UID: \"b98d9165-4521-4f00-9bf4-3ed9da4f53d8\") " pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.565894 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.565845 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75kr7\" (UniqueName: \"kubernetes.io/projected/05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3-kube-api-access-75kr7\") pod \"multus-sl99s\" (UID: \"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3\") " pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.667752 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.667733 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2jddj" Jun 09 01:15:40.675586 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:40.675567 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc85ec566_05db_4e9c_a563_0615d3a76f06.slice/crio-e5b74f8825d6122c22b16a6b11be8f1dc4f5e037005c487f46524ea15f6ae00b WatchSource:0}: Error finding container e5b74f8825d6122c22b16a6b11be8f1dc4f5e037005c487f46524ea15f6ae00b: Status 404 returned error can't find the container with id e5b74f8825d6122c22b16a6b11be8f1dc4f5e037005c487f46524ea15f6ae00b Jun 09 01:15:40.691412 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.691395 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" Jun 09 01:15:40.697718 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:40.697701 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod744d9eed_638b_4a43_a4d6_e118377f5139.slice/crio-49aa7ec50823886bd4a51f7ca6bb4d875ddd4d6755bffbd4e3c8adc216348aa8 WatchSource:0}: Error finding container 49aa7ec50823886bd4a51f7ca6bb4d875ddd4d6755bffbd4e3c8adc216348aa8: Status 404 returned error can't find the container with id 49aa7ec50823886bd4a51f7ca6bb4d875ddd4d6755bffbd4e3c8adc216348aa8 Jun 09 01:15:40.698879 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.698848 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rs95s" Jun 09 01:15:40.705241 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.705214 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-87mlr" Jun 09 01:15:40.705475 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:40.705457 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e1cf77d_cef0_4363_a02f_f0887da4e984.slice/crio-54add565ef9b6155094be99facfc88ef9eb6b5e8c18b1390cc76a6da0bd5e881 WatchSource:0}: Error finding container 54add565ef9b6155094be99facfc88ef9eb6b5e8c18b1390cc76a6da0bd5e881: Status 404 returned error can't find the container with id 54add565ef9b6155094be99facfc88ef9eb6b5e8c18b1390cc76a6da0bd5e881 Jun 09 01:15:40.710443 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.710390 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5zrlt" Jun 09 01:15:40.712216 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:40.712197 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4108bdff_ba06_42ab_9b64_a7be1cf9c8a4.slice/crio-79d9904871c5585179f602a3da8523c1f659dc1fbd6e6a6d7c0072e3da57cd6e WatchSource:0}: Error finding container 79d9904871c5585179f602a3da8523c1f659dc1fbd6e6a6d7c0072e3da57cd6e: Status 404 returned error can't find the container with id 79d9904871c5585179f602a3da8523c1f659dc1fbd6e6a6d7c0072e3da57cd6e Jun 09 01:15:40.716067 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.716050 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:15:40.717470 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:40.717451 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd5c78f_a01c_41d6_81ce_fd01b8b78e43.slice/crio-6b229dba45b269446f1bc010b7ae6e43ae3bf36c91cc762ed51a2dcb27e5a193 WatchSource:0}: Error finding container 6b229dba45b269446f1bc010b7ae6e43ae3bf36c91cc762ed51a2dcb27e5a193: Status 404 returned error can't find the container with id 6b229dba45b269446f1bc010b7ae6e43ae3bf36c91cc762ed51a2dcb27e5a193 Jun 09 01:15:40.723461 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.723437 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xlqwl" Jun 09 01:15:40.723868 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:40.723843 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3a9620d_b3b8_48ae_b007_acd921f2aeac.slice/crio-ab2c60b4989c9cb4e6d39af77ecbc0f33d151a84c78ce2fb706fe5be15826dfe WatchSource:0}: Error finding container ab2c60b4989c9cb4e6d39af77ecbc0f33d151a84c78ce2fb706fe5be15826dfe: Status 404 returned error can't find the container with id ab2c60b4989c9cb4e6d39af77ecbc0f33d151a84c78ce2fb706fe5be15826dfe Jun 09 01:15:40.728508 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.728482 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" Jun 09 01:15:40.733327 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.733194 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-sl99s" Jun 09 01:15:40.736878 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:40.736857 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod268a2882_9374_4669_8ed7_fefeee12a63e.slice/crio-50bb47715c6c86739bdb32a7467df33f9c55795fe65e9916ffe44b802c91eec8 WatchSource:0}: Error finding container 50bb47715c6c86739bdb32a7467df33f9c55795fe65e9916ffe44b802c91eec8: Status 404 returned error can't find the container with id 50bb47715c6c86739bdb32a7467df33f9c55795fe65e9916ffe44b802c91eec8 Jun 09 01:15:40.748451 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:15:40.746833 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb98d9165_4521_4f00_9bf4_3ed9da4f53d8.slice/crio-a9a630968b9038240e5426e09a1b8a23a67b590e4b82afdb1c0d05cc64f71550 WatchSource:0}: Error finding container a9a630968b9038240e5426e09a1b8a23a67b590e4b82afdb1c0d05cc64f71550: Status 404 returned error can't find the container with id a9a630968b9038240e5426e09a1b8a23a67b590e4b82afdb1c0d05cc64f71550 Jun 09 01:15:40.961773 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:40.961703 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs\") pod \"network-metrics-daemon-8prh2\" (UID: \"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19\") " pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:40.961901 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:40.961824 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 01:15:40.961901 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:40.961882 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs podName:8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19 nodeName:}" failed. No retries permitted until 2026-06-09 01:15:41.961864281 +0000 UTC m=+2.980168456 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs") pod "network-metrics-daemon-8prh2" (UID: "8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 01:15:41.063168 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:41.062380 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret\") pod \"global-pull-secret-syncer-wgbm8\" (UID: \"76027187-053a-4879-b929-163968a1cd82\") " pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:41.063168 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:41.062442 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r62gc\" (UniqueName: \"kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc\") pod \"network-check-target-s4jjf\" (UID: \"42e3c300-dc21-44d9-a8a1-6cd16b11dc77\") " pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:15:41.063168 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:41.062610 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 01:15:41.063168 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:41.062643 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 01:15:41.063168 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:41.062657 2555 projected.go:196] Error preparing data for projected volume kube-api-access-r62gc for pod openshift-network-diagnostics/network-check-target-s4jjf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 01:15:41.063168 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:41.062712 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc podName:42e3c300-dc21-44d9-a8a1-6cd16b11dc77 nodeName:}" failed. No retries permitted until 2026-06-09 01:15:42.062693032 +0000 UTC m=+3.080997209 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r62gc" (UniqueName: "kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc") pod "network-check-target-s4jjf" (UID: "42e3c300-dc21-44d9-a8a1-6cd16b11dc77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 01:15:41.063168 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:41.063079 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 01:15:41.063168 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:41.063131 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret podName:76027187-053a-4879-b929-163968a1cd82 nodeName:}" failed. No retries permitted until 2026-06-09 01:15:42.063115443 +0000 UTC m=+3.081419621 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret") pod "global-pull-secret-syncer-wgbm8" (UID: "76027187-053a-4879-b929-163968a1cd82") : object "kube-system"/"original-pull-secret" not registered Jun 09 01:15:41.124713 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:41.124519 2555 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 01:15:41.398045 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:41.397943 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-08 01:10:40 +0000 UTC" deadline="2027-12-15 12:07:30.755690562 +0000 UTC" Jun 09 01:15:41.398045 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:41.397993 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13306h51m49.357700074s" Jun 09 01:15:41.437268 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:41.437175 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" event={"ID":"b98d9165-4521-4f00-9bf4-3ed9da4f53d8","Type":"ContainerStarted","Data":"a9a630968b9038240e5426e09a1b8a23a67b590e4b82afdb1c0d05cc64f71550"} Jun 09 01:15:41.453565 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:41.453459 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xlqwl" event={"ID":"268a2882-9374-4669-8ed7-fefeee12a63e","Type":"ContainerStarted","Data":"50bb47715c6c86739bdb32a7467df33f9c55795fe65e9916ffe44b802c91eec8"} Jun 09 01:15:41.468146 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:41.468118 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" event={"ID":"a3a9620d-b3b8-48ae-b007-acd921f2aeac","Type":"ContainerStarted","Data":"ab2c60b4989c9cb4e6d39af77ecbc0f33d151a84c78ce2fb706fe5be15826dfe"} Jun 09 01:15:41.502346 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:41.502111 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zrlt" event={"ID":"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43","Type":"ContainerStarted","Data":"6b229dba45b269446f1bc010b7ae6e43ae3bf36c91cc762ed51a2dcb27e5a193"} Jun 09 01:15:41.508132 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:41.508066 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-87mlr" event={"ID":"4108bdff-ba06-42ab-9b64-a7be1cf9c8a4","Type":"ContainerStarted","Data":"79d9904871c5585179f602a3da8523c1f659dc1fbd6e6a6d7c0072e3da57cd6e"} Jun 09 01:15:41.526562 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:41.526526 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2jddj" event={"ID":"c85ec566-05db-4e9c-a563-0615d3a76f06","Type":"ContainerStarted","Data":"e5b74f8825d6122c22b16a6b11be8f1dc4f5e037005c487f46524ea15f6ae00b"} Jun 09 01:15:41.555232 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:41.555205 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sl99s" event={"ID":"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3","Type":"ContainerStarted","Data":"a2b84be5ffb0d8b601c9fb0f85bb13f1ea0ca0d6066d835bee3d5f7b7a46219f"} Jun 09 01:15:41.565914 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:41.565888 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rs95s" event={"ID":"8e1cf77d-cef0-4363-a02f-f0887da4e984","Type":"ContainerStarted","Data":"54add565ef9b6155094be99facfc88ef9eb6b5e8c18b1390cc76a6da0bd5e881"} Jun 09 01:15:41.576895 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:41.576867 2555 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 01:15:41.598876 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:41.598855 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" event={"ID":"744d9eed-638b-4a43-a4d6-e118377f5139","Type":"ContainerStarted","Data":"49aa7ec50823886bd4a51f7ca6bb4d875ddd4d6755bffbd4e3c8adc216348aa8"} Jun 09 01:15:41.851831 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:41.851762 2555 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 01:15:41.971554 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:41.971511 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs\") pod \"network-metrics-daemon-8prh2\" (UID: \"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19\") " pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:41.971704 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:41.971680 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 01:15:41.971759 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:41.971736 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs podName:8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19 nodeName:}" failed. No retries permitted until 2026-06-09 01:15:43.971718246 +0000 UTC m=+4.990022433 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs") pod "network-metrics-daemon-8prh2" (UID: "8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 01:15:42.071839 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:42.071811 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r62gc\" (UniqueName: \"kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc\") pod \"network-check-target-s4jjf\" (UID: \"42e3c300-dc21-44d9-a8a1-6cd16b11dc77\") " pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:15:42.071993 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:42.071860 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret\") pod \"global-pull-secret-syncer-wgbm8\" (UID: \"76027187-053a-4879-b929-163968a1cd82\") " pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:42.071993 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:42.071945 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 01:15:42.071993 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:42.071989 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret podName:76027187-053a-4879-b929-163968a1cd82 nodeName:}" failed. No retries permitted until 2026-06-09 01:15:44.071975527 +0000 UTC m=+5.090279698 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret") pod "global-pull-secret-syncer-wgbm8" (UID: "76027187-053a-4879-b929-163968a1cd82") : object "kube-system"/"original-pull-secret" not registered Jun 09 01:15:42.072391 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:42.072367 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 01:15:42.072477 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:42.072401 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 01:15:42.072477 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:42.072414 2555 projected.go:196] Error preparing data for projected volume kube-api-access-r62gc for pod openshift-network-diagnostics/network-check-target-s4jjf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 01:15:42.072477 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:42.072465 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc podName:42e3c300-dc21-44d9-a8a1-6cd16b11dc77 nodeName:}" failed. No retries permitted until 2026-06-09 01:15:44.072449216 +0000 UTC m=+5.090753390 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r62gc" (UniqueName: "kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc") pod "network-check-target-s4jjf" (UID: "42e3c300-dc21-44d9-a8a1-6cd16b11dc77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 01:15:42.398660 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:42.398573 2555 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-08 01:10:40 +0000 UTC" deadline="2027-11-09 06:53:51.088429511 +0000 UTC" Jun 09 01:15:42.398660 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:42.398609 2555 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12437h38m8.689825092s" Jun 09 01:15:42.410535 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:42.409749 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:42.410535 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:42.409874 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:15:42.410535 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:42.410271 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:42.410535 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:42.410360 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:15:42.410535 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:42.410417 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:15:42.410535 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:42.410481 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:15:43.986374 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:43.986335 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs\") pod \"network-metrics-daemon-8prh2\" (UID: \"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19\") " pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:43.986805 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:43.986509 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 01:15:43.986805 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:43.986604 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs podName:8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19 nodeName:}" failed. No retries permitted until 2026-06-09 01:15:47.986580775 +0000 UTC m=+9.004884960 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs") pod "network-metrics-daemon-8prh2" (UID: "8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 01:15:44.087227 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:44.087192 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret\") pod \"global-pull-secret-syncer-wgbm8\" (UID: \"76027187-053a-4879-b929-163968a1cd82\") " pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:44.087395 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:44.087262 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r62gc\" (UniqueName: \"kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc\") pod \"network-check-target-s4jjf\" (UID: \"42e3c300-dc21-44d9-a8a1-6cd16b11dc77\") " pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:15:44.087462 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:44.087421 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 01:15:44.087462 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:44.087440 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 01:15:44.087462 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:44.087454 2555 projected.go:196] Error preparing data for projected volume kube-api-access-r62gc for pod openshift-network-diagnostics/network-check-target-s4jjf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 01:15:44.087631 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:44.087509 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc podName:42e3c300-dc21-44d9-a8a1-6cd16b11dc77 nodeName:}" failed. No retries permitted until 2026-06-09 01:15:48.087491731 +0000 UTC m=+9.105795909 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r62gc" (UniqueName: "kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc") pod "network-check-target-s4jjf" (UID: "42e3c300-dc21-44d9-a8a1-6cd16b11dc77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 01:15:44.087926 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:44.087906 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 01:15:44.087993 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:44.087965 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret podName:76027187-053a-4879-b929-163968a1cd82 nodeName:}" failed. No retries permitted until 2026-06-09 01:15:48.087949722 +0000 UTC m=+9.106253899 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret") pod "global-pull-secret-syncer-wgbm8" (UID: "76027187-053a-4879-b929-163968a1cd82") : object "kube-system"/"original-pull-secret" not registered Jun 09 01:15:44.410330 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:44.410244 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:44.410495 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:44.410368 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:15:44.410726 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:44.410707 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:44.410806 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:44.410736 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:15:44.410856 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:44.410817 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:15:44.410907 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:44.410888 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:15:46.409961 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:46.409476 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:46.409961 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:46.409625 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:15:46.409961 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:46.409743 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:46.409961 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:46.409784 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:15:46.409961 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:46.409864 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:15:46.409961 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:46.409945 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:15:48.019838 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:48.019710 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs\") pod \"network-metrics-daemon-8prh2\" (UID: \"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19\") " pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:48.020247 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:48.019880 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 01:15:48.020247 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:48.019944 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs podName:8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19 nodeName:}" failed. No retries permitted until 2026-06-09 01:15:56.019922128 +0000 UTC m=+17.038226304 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs") pod "network-metrics-daemon-8prh2" (UID: "8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 01:15:48.121400 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:48.121008 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret\") pod \"global-pull-secret-syncer-wgbm8\" (UID: \"76027187-053a-4879-b929-163968a1cd82\") " pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:48.121400 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:48.121074 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r62gc\" (UniqueName: \"kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc\") pod \"network-check-target-s4jjf\" (UID: \"42e3c300-dc21-44d9-a8a1-6cd16b11dc77\") " pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:15:48.121400 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:48.121266 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 01:15:48.121400 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:48.121290 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 01:15:48.121400 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:48.121308 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 01:15:48.121400 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:48.121320 2555 projected.go:196] Error preparing data for projected volume kube-api-access-r62gc for pod openshift-network-diagnostics/network-check-target-s4jjf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 01:15:48.121400 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:48.121329 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret podName:76027187-053a-4879-b929-163968a1cd82 nodeName:}" failed. No retries permitted until 2026-06-09 01:15:56.121310422 +0000 UTC m=+17.139614595 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret") pod "global-pull-secret-syncer-wgbm8" (UID: "76027187-053a-4879-b929-163968a1cd82") : object "kube-system"/"original-pull-secret" not registered Jun 09 01:15:48.121400 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:48.121373 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc podName:42e3c300-dc21-44d9-a8a1-6cd16b11dc77 nodeName:}" failed. No retries permitted until 2026-06-09 01:15:56.121357296 +0000 UTC m=+17.139661481 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r62gc" (UniqueName: "kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc") pod "network-check-target-s4jjf" (UID: "42e3c300-dc21-44d9-a8a1-6cd16b11dc77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 01:15:48.410335 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:48.410257 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:48.410505 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:48.410268 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:48.410505 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:48.410395 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:15:48.410505 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:48.410454 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:15:48.410505 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:48.410267 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:15:48.410748 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:48.410574 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:15:50.410153 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:50.410119 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:50.410153 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:50.410149 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:15:50.410772 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:50.410149 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:50.410772 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:50.410239 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:15:50.410772 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:50.410314 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:15:50.410772 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:50.410394 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:15:52.410353 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:52.410322 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:52.410802 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:52.410423 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:15:52.410802 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:52.410438 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:15:52.410802 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:52.410462 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:52.410802 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:52.410591 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:15:52.410802 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:52.410704 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:15:54.409675 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:54.409596 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:54.409675 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:54.409630 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:15:54.410147 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:54.409706 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:54.410147 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:54.409722 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:15:54.410147 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:54.409800 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:15:54.410147 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:54.409870 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:15:56.087045 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:56.086963 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs\") pod \"network-metrics-daemon-8prh2\" (UID: \"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19\") " pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:56.087471 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:56.087106 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 01:15:56.087471 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:56.087167 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs podName:8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19 nodeName:}" failed. No retries permitted until 2026-06-09 01:16:12.087148885 +0000 UTC m=+33.105453060 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs") pod "network-metrics-daemon-8prh2" (UID: "8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 01:15:56.187693 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:56.187658 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret\") pod \"global-pull-secret-syncer-wgbm8\" (UID: \"76027187-053a-4879-b929-163968a1cd82\") " pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:56.187835 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:56.187715 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r62gc\" (UniqueName: \"kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc\") pod \"network-check-target-s4jjf\" (UID: \"42e3c300-dc21-44d9-a8a1-6cd16b11dc77\") " pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:15:56.187835 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:56.187817 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 01:15:56.187948 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:56.187836 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 01:15:56.187948 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:56.187852 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 01:15:56.187948 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:56.187862 2555 projected.go:196] Error preparing data for projected volume kube-api-access-r62gc for pod openshift-network-diagnostics/network-check-target-s4jjf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 01:15:56.187948 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:56.187891 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret podName:76027187-053a-4879-b929-163968a1cd82 nodeName:}" failed. No retries permitted until 2026-06-09 01:16:12.187871528 +0000 UTC m=+33.206175716 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret") pod "global-pull-secret-syncer-wgbm8" (UID: "76027187-053a-4879-b929-163968a1cd82") : object "kube-system"/"original-pull-secret" not registered Jun 09 01:15:56.187948 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:56.187912 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc podName:42e3c300-dc21-44d9-a8a1-6cd16b11dc77 nodeName:}" failed. No retries permitted until 2026-06-09 01:16:12.187903133 +0000 UTC m=+33.206207303 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r62gc" (UniqueName: "kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc") pod "network-check-target-s4jjf" (UID: "42e3c300-dc21-44d9-a8a1-6cd16b11dc77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 01:15:56.409586 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:56.409500 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:56.409586 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:56.409518 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:15:56.409586 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:56.409519 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:56.409830 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:56.409625 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:15:56.409830 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:56.409699 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:15:56.409830 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:56.409774 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:15:58.409350 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:58.409316 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:15:58.409350 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:58.409353 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:15:58.409939 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:15:58.409353 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:15:58.409939 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:58.409439 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:15:58.409939 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:58.409535 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:15:58.409939 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:15:58.409625 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:16:00.410135 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:00.410109 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:16:00.410461 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:00.410200 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:16:00.410461 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:00.410217 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:16:00.410461 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:00.410240 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:16:00.410461 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:00.410328 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:16:00.410461 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:00.410428 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:16:00.639270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:00.639146 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal" event={"ID":"a29d25d72d9ebc3da303cb0225e4c752","Type":"ContainerStarted","Data":"8849c39e26bc86377031f3af0a746beed5659cadad147e420e19eaa7ec33b9ff"} Jun 09 01:16:01.643745 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:01.643672 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" event={"ID":"b98d9165-4521-4f00-9bf4-3ed9da4f53d8","Type":"ContainerStarted","Data":"6821dc36a79c6649cc7afa28deb5b28557d9b64b7a614a20a93dd4e1bb38d4d4"} Jun 09 01:16:01.647184 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:01.647165 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:16:01.647520 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:01.647497 2555 generic.go:356] "Generic (PLEG): container finished" podID="a3a9620d-b3b8-48ae-b007-acd921f2aeac" containerID="e5f34ed47346515484715fe928e7caabb6ef1755b95dd3abc600eb5d4f92a381" exitCode=1 Jun 09 01:16:01.647629 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:01.647571 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" event={"ID":"a3a9620d-b3b8-48ae-b007-acd921f2aeac","Type":"ContainerStarted","Data":"d057a8da7a73e860fd3367141f8db87714e4d89fdbfc53e3cc0e488a7a6abbe0"} Jun 09 01:16:01.647676 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:01.647641 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" event={"ID":"a3a9620d-b3b8-48ae-b007-acd921f2aeac","Type":"ContainerStarted","Data":"b3d8772dae6799e3a90d9d0be1b9dab534f5e4785d4caf80b98c7899d2a49c09"} Jun 09 01:16:01.647710 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:01.647680 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" event={"ID":"a3a9620d-b3b8-48ae-b007-acd921f2aeac","Type":"ContainerStarted","Data":"8954cc6e01dfa78e9410719580910a4172374b87ec88fc4805a08f1dc68ad18f"} Jun 09 01:16:01.647752 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:01.647724 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" event={"ID":"a3a9620d-b3b8-48ae-b007-acd921f2aeac","Type":"ContainerStarted","Data":"caa70a2bd14c197fdf9166406f1b91d37dee78fc1d7acdde168bf86a63688fd5"} Jun 09 01:16:01.647788 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:01.647755 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" event={"ID":"a3a9620d-b3b8-48ae-b007-acd921f2aeac","Type":"ContainerDied","Data":"e5f34ed47346515484715fe928e7caabb6ef1755b95dd3abc600eb5d4f92a381"} Jun 09 01:16:01.647788 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:01.647783 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" event={"ID":"a3a9620d-b3b8-48ae-b007-acd921f2aeac","Type":"ContainerStarted","Data":"81359ee2ccd46d1cfc41bfb5cd7c7edbf70f60122cb9b794019b8a5679313e82"} Jun 09 01:16:01.648979 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:01.648959 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-sl99s" event={"ID":"05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3","Type":"ContainerStarted","Data":"eab5d80b30fda97e1b1035cee1a4599a61c5a4b4f66feddf40b63eadffc26f8f"} Jun 09 01:16:01.660640 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:01.660601 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nfwxf" podStartSLOduration=2.9531566209999998 podStartE2EDuration="22.660588963s" totalImagesPullingTime="19.707432342s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 01:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:16:01.659897099 +0000 UTC m=+22.678201293" watchObservedRunningTime="2026-06-09 01:16:01.660588963 +0000 UTC m=+22.678893173" Jun 09 01:16:01.673778 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:01.673738 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal" podStartSLOduration=22.673721108 podStartE2EDuration="22.673721108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 01:15:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:16:01.673502366 +0000 UTC m=+22.691806560" watchObservedRunningTime="2026-06-09 01:16:01.673721108 +0000 UTC m=+22.692025302" Jun 09 01:16:01.691061 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:01.691017 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-sl99s" podStartSLOduration=2.9722136089999998 podStartE2EDuration="22.690990445s" totalImagesPullingTime="19.718776836s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 01:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:16:01.690394094 +0000 UTC m=+22.708698287" watchObservedRunningTime="2026-06-09 01:16:01.690990445 +0000 UTC m=+22.709294639" Jun 09 01:16:02.410221 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:02.410194 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:16:02.410438 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:02.410194 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:16:02.410438 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:02.410294 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:16:02.410438 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:02.410415 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:16:02.410599 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:02.410194 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:16:02.410599 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:02.410537 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:16:02.651778 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:02.651742 2555 generic.go:356] "Generic (PLEG): container finished" podID="2018821b48cefdd51ae3152836bd5175" containerID="a2524434526d939d8738735be8e57987f04ef34ad008dde87b5c4f75c873a3aa" exitCode=0 Jun 09 01:16:02.652502 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:02.651828 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" event={"ID":"2018821b48cefdd51ae3152836bd5175","Type":"ContainerDied","Data":"a2524434526d939d8738735be8e57987f04ef34ad008dde87b5c4f75c873a3aa"} Jun 09 01:16:02.653324 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:02.653282 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xlqwl" event={"ID":"268a2882-9374-4669-8ed7-fefeee12a63e","Type":"ContainerStarted","Data":"1e6ccc40908bb8b5a23c7bbfe66891c155213e41aec3cd41899d565d29666506"} Jun 09 01:16:02.654612 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:02.654592 2555 generic.go:356] "Generic (PLEG): container finished" podID="fbd5c78f-a01c-41d6-81ce-fd01b8b78e43" containerID="ab493a16fe82d907cdec87371276755d6889bc9ad6cc784bd922c9e9edf17051" exitCode=0 Jun 09 01:16:02.654723 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:02.654674 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zrlt" event={"ID":"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43","Type":"ContainerDied","Data":"ab493a16fe82d907cdec87371276755d6889bc9ad6cc784bd922c9e9edf17051"} Jun 09 01:16:02.656104 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:02.656085 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-87mlr" event={"ID":"4108bdff-ba06-42ab-9b64-a7be1cf9c8a4","Type":"ContainerStarted","Data":"ffe71e2f23c4cebb3be88d2ea5165a39aa38d3fd47fa28e922a78ee51a9b7309"} Jun 09 01:16:02.657592 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:02.657569 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2jddj" event={"ID":"c85ec566-05db-4e9c-a563-0615d3a76f06","Type":"ContainerStarted","Data":"ce1624d28165d92e6bc1a06264f1db11b2cac5e9f3a4c1caa035bbcd081e2748"} Jun 09 01:16:02.659003 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:02.658973 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rs95s" event={"ID":"8e1cf77d-cef0-4363-a02f-f0887da4e984","Type":"ContainerStarted","Data":"fb17361a295de37a5918c077ea410b2f8052eea4a07b38ff41edd4d04bee36b3"} Jun 09 01:16:02.660259 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:02.660215 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" event={"ID":"744d9eed-638b-4a43-a4d6-e118377f5139","Type":"ContainerStarted","Data":"d53944a82ef700ab2e65cc33d6774f3d6062aef385a8e8403eccfdcd0e75e007"} Jun 09 01:16:02.680926 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:02.680886 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2jddj" podStartSLOduration=3.900046064 podStartE2EDuration="23.680873115s" totalImagesPullingTime="19.780827051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 01:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:16:02.680249782 +0000 UTC m=+23.698554002" watchObservedRunningTime="2026-06-09 01:16:02.680873115 +0000 UTC m=+23.699177308" Jun 09 01:16:02.695587 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:02.695534 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rs95s" podStartSLOduration=3.943068006 podStartE2EDuration="23.695519428s" totalImagesPullingTime="19.752451422s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 01:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:16:02.695481169 +0000 UTC m=+23.713785374" watchObservedRunningTime="2026-06-09 01:16:02.695519428 +0000 UTC m=+23.713823622" Jun 09 01:16:02.709158 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:02.709123 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-87mlr" podStartSLOduration=3.9893550859999998 podStartE2EDuration="23.709112932s" totalImagesPullingTime="19.719757846s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 01:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:16:02.709098416 +0000 UTC m=+23.727402608" watchObservedRunningTime="2026-06-09 01:16:02.709112932 +0000 UTC m=+23.727417124" Jun 09 01:16:02.744258 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:02.744227 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xlqwl" podStartSLOduration=4.049227366 podStartE2EDuration="23.744216931s" totalImagesPullingTime="19.694989565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:16:02.744074764 +0000 UTC m=+23.762378945" watchObservedRunningTime="2026-06-09 01:16:02.744216931 +0000 UTC m=+23.762521114" Jun 09 01:16:02.888907 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:02.888882 2555 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 09 01:16:03.411281 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:03.411117 2555 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-09T01:16:02.888902765Z","UUID":"f457492f-3bc2-44e0-bbb4-a89e1a6a3c00","Handler":null,"Name":"","Endpoint":""} Jun 09 01:16:03.413518 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:03.413213 2555 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 09 01:16:03.413518 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:03.413258 2555 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 09 01:16:03.665852 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:03.665447 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" event={"ID":"744d9eed-638b-4a43-a4d6-e118377f5139","Type":"ContainerStarted","Data":"ffc113bdb41a3da7a04bc51f9f03af0c1bafb85224a925f4898b4414cd698d9f"} Jun 09 01:16:03.667955 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:03.667896 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" event={"ID":"2018821b48cefdd51ae3152836bd5175","Type":"ContainerStarted","Data":"15e3c78637ff68af8274a1b4a3056674b40c25ee5d9a6103a785e9ab6ada4ac6"} Jun 09 01:16:03.671608 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:03.671498 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:16:03.672046 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:03.672021 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" event={"ID":"a3a9620d-b3b8-48ae-b007-acd921f2aeac","Type":"ContainerStarted","Data":"9a1da2cbd948a8ebc401b8c0adeef7409bedbc5c4057243f49f6cdb567758e00"} Jun 09 01:16:03.683825 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:03.683775 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" podStartSLOduration=24.683757446 podStartE2EDuration="24.683757446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 01:15:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:16:03.683008657 +0000 UTC m=+24.701312850" watchObservedRunningTime="2026-06-09 01:16:03.683757446 +0000 UTC m=+24.702061640" Jun 09 01:16:04.409835 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:04.409765 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:16:04.409835 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:04.409808 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:16:04.410038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:04.409771 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:16:04.410038 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:04.409884 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:16:04.410038 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:04.409966 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:16:04.410191 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:04.410053 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:16:04.675646 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:04.675583 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" event={"ID":"744d9eed-638b-4a43-a4d6-e118377f5139","Type":"ContainerStarted","Data":"c1f762e57948c62d5fa364682370eeb1889198bf8c0fe6ed675ead35ce4fcc59"} Jun 09 01:16:04.696587 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:04.696518 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjlt9" podStartSLOduration=3.987929989 podStartE2EDuration="25.6964996s" totalImagesPullingTime="21.708569611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 01:15:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:16:04.695774411 +0000 UTC m=+25.714078602" watchObservedRunningTime="2026-06-09 01:16:04.6964996 +0000 UTC m=+25.714803793" Jun 09 01:16:06.409959 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:06.409926 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:16:06.410402 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:06.409939 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:16:06.410402 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:06.410045 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:16:06.410402 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:06.410070 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:16:06.410577 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:06.410482 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:16:06.410660 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:06.410638 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:16:06.573536 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:06.573498 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xlqwl" Jun 09 01:16:06.574260 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:06.574239 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xlqwl" Jun 09 01:16:06.679423 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:06.679360 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xlqwl" Jun 09 01:16:06.679860 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:06.679839 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xlqwl" Jun 09 01:16:07.682966 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:07.682771 2555 generic.go:356] "Generic (PLEG): container finished" podID="fbd5c78f-a01c-41d6-81ce-fd01b8b78e43" containerID="bf5dc8e4f2e5d3c3c4e60ff8368515c9a3d3afcc9dd53ee7185e42ed38c98c05" exitCode=0 Jun 09 01:16:07.683299 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:07.682859 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zrlt" event={"ID":"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43","Type":"ContainerDied","Data":"bf5dc8e4f2e5d3c3c4e60ff8368515c9a3d3afcc9dd53ee7185e42ed38c98c05"} Jun 09 01:16:07.686766 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:07.686751 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:16:07.687127 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:07.687107 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" event={"ID":"a3a9620d-b3b8-48ae-b007-acd921f2aeac","Type":"ContainerStarted","Data":"3a8fdccbcba10f695c54d9a414683ce9074e49ebde8ffdd550fe59656ace3854"} Jun 09 01:16:07.687362 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:07.687335 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:16:07.687438 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:07.687412 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:16:07.687514 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:07.687502 2555 scope.go:117] "RemoveContainer" containerID="e5f34ed47346515484715fe928e7caabb6ef1755b95dd3abc600eb5d4f92a381" Jun 09 01:16:07.702255 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:07.702236 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:16:08.410092 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:08.410060 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:16:08.410294 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:08.410105 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:16:08.410294 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:08.410074 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:16:08.410294 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:08.410188 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:16:08.410458 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:08.410326 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:16:08.410458 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:08.410396 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:16:08.692688 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:08.692628 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:16:08.693142 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:08.692990 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" event={"ID":"a3a9620d-b3b8-48ae-b007-acd921f2aeac","Type":"ContainerStarted","Data":"2cc81c544d1f70bc84460b207820e16c760f8ab624de19d063df7ae25c729075"} Jun 09 01:16:08.693312 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:08.693290 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:16:08.706352 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:08.706334 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:16:08.719968 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:08.719927 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" podStartSLOduration=9.905375584 podStartE2EDuration="29.719914177s" totalImagesPullingTime="19.814538593s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 01:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:16:08.718240571 +0000 UTC m=+29.736544828" watchObservedRunningTime="2026-06-09 01:16:08.719914177 +0000 UTC m=+29.738218378" Jun 09 01:16:09.141204 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:09.141176 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-s4jjf"] Jun 09 01:16:09.141336 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:09.141292 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:16:09.141416 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:09.141392 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:16:09.144684 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:09.144659 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wgbm8"] Jun 09 01:16:09.144808 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:09.144771 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:16:09.144893 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:09.144868 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:16:09.145304 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:09.145282 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8prh2"] Jun 09 01:16:09.145401 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:09.145387 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:16:09.145514 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:09.145494 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:16:09.697015 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:09.696983 2555 generic.go:356] "Generic (PLEG): container finished" podID="fbd5c78f-a01c-41d6-81ce-fd01b8b78e43" containerID="6a01bd9c280e335e05432652d5253fa48106b0f8c4eb5715c48df3122c43c7a4" exitCode=0 Jun 09 01:16:09.697708 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:09.697071 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zrlt" event={"ID":"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43","Type":"ContainerDied","Data":"6a01bd9c280e335e05432652d5253fa48106b0f8c4eb5715c48df3122c43c7a4"} Jun 09 01:16:10.410012 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:10.409847 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:16:10.410137 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:10.410087 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:16:10.701527 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:10.701501 2555 generic.go:356] "Generic (PLEG): container finished" podID="fbd5c78f-a01c-41d6-81ce-fd01b8b78e43" containerID="063807a41af09725f53b97fb84b824266f394edcea6368f882e61fa2630b7d42" exitCode=0 Jun 09 01:16:10.701791 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:10.701556 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zrlt" event={"ID":"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43","Type":"ContainerDied","Data":"063807a41af09725f53b97fb84b824266f394edcea6368f882e61fa2630b7d42"} Jun 09 01:16:11.409638 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:11.409616 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:16:11.409745 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:11.409696 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:16:11.409805 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:11.409789 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:16:11.409912 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:11.409889 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:16:12.113831 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:12.113799 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs\") pod \"network-metrics-daemon-8prh2\" (UID: \"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19\") " pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:16:12.114252 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:12.113931 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 01:16:12.114252 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:12.114001 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs podName:8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19 nodeName:}" failed. No retries permitted until 2026-06-09 01:16:44.113980875 +0000 UTC m=+65.132285062 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs") pod "network-metrics-daemon-8prh2" (UID: "8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 01:16:12.214687 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:12.214659 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret\") pod \"global-pull-secret-syncer-wgbm8\" (UID: \"76027187-053a-4879-b929-163968a1cd82\") " pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:16:12.214799 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:12.214713 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r62gc\" (UniqueName: \"kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc\") pod \"network-check-target-s4jjf\" (UID: \"42e3c300-dc21-44d9-a8a1-6cd16b11dc77\") " pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:16:12.214799 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:12.214785 2555 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 01:16:12.214870 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:12.214827 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 01:16:12.214870 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:12.214846 2555 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 01:16:12.214870 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:12.214841 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret podName:76027187-053a-4879-b929-163968a1cd82 nodeName:}" failed. No retries permitted until 2026-06-09 01:16:44.214824707 +0000 UTC m=+65.233128880 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret") pod "global-pull-secret-syncer-wgbm8" (UID: "76027187-053a-4879-b929-163968a1cd82") : object "kube-system"/"original-pull-secret" not registered Jun 09 01:16:12.214870 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:12.214861 2555 projected.go:196] Error preparing data for projected volume kube-api-access-r62gc for pod openshift-network-diagnostics/network-check-target-s4jjf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 01:16:12.215037 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:12.214910 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc podName:42e3c300-dc21-44d9-a8a1-6cd16b11dc77 nodeName:}" failed. No retries permitted until 2026-06-09 01:16:44.214896024 +0000 UTC m=+65.233200217 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-r62gc" (UniqueName: "kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc") pod "network-check-target-s4jjf" (UID: "42e3c300-dc21-44d9-a8a1-6cd16b11dc77") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 01:16:12.410357 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:12.410304 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:16:12.410458 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:12.410410 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:16:13.410429 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:13.410401 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:16:13.410776 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:13.410401 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:16:13.410776 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:13.410501 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:16:13.410776 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:13.410582 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s4jjf" podUID="42e3c300-dc21-44d9-a8a1-6cd16b11dc77" Jun 09 01:16:14.409287 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:14.409265 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:16:14.409420 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:14.409374 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wgbm8" podUID="76027187-053a-4879-b929-163968a1cd82" Jun 09 01:16:14.817843 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:14.817819 2555 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeReady" Jun 09 01:16:14.818157 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:14.817908 2555 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 09 01:16:14.861819 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:14.861794 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5vfr5"] Jun 09 01:16:14.881265 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:14.881238 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-prpjp"] Jun 09 01:16:14.881409 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:14.881347 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:14.884167 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:14.884143 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 09 01:16:14.884273 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:14.884171 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j9cpl\"" Jun 09 01:16:14.884273 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:14.884184 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 09 01:16:14.905286 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:14.905265 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5vfr5"] Jun 09 01:16:14.905384 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:14.905300 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-prpjp"] Jun 09 01:16:14.905384 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:14.905381 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-prpjp" Jun 09 01:16:14.908277 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:14.908260 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-7cpv6\"" Jun 09 01:16:14.908380 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:14.908299 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 09 01:16:14.908707 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:14.908578 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 09 01:16:14.908707 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:14.908630 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 09 01:16:15.037325 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.037301 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-269lb\" (UniqueName: \"kubernetes.io/projected/c4cca482-39bd-4a22-bed7-e85a83b32b47-kube-api-access-269lb\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:15.037424 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.037341 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e0bcec3-73e6-4a2a-bc46-2848c31d9ce9-cert\") pod \"ingress-canary-prpjp\" (UID: \"5e0bcec3-73e6-4a2a-bc46-2848c31d9ce9\") " pod="openshift-ingress-canary/ingress-canary-prpjp" Jun 09 01:16:15.037424 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.037362 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4cca482-39bd-4a22-bed7-e85a83b32b47-tmp-dir\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:15.037424 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.037379 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c4cca482-39bd-4a22-bed7-e85a83b32b47-config-volume\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:15.037424 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.037397 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmsss\" (UniqueName: \"kubernetes.io/projected/5e0bcec3-73e6-4a2a-bc46-2848c31d9ce9-kube-api-access-dmsss\") pod \"ingress-canary-prpjp\" (UID: \"5e0bcec3-73e6-4a2a-bc46-2848c31d9ce9\") " pod="openshift-ingress-canary/ingress-canary-prpjp" Jun 09 01:16:15.037424 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.037425 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:15.138727 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.138672 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:15.138824 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.138726 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-269lb\" (UniqueName: \"kubernetes.io/projected/c4cca482-39bd-4a22-bed7-e85a83b32b47-kube-api-access-269lb\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:15.138824 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.138780 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e0bcec3-73e6-4a2a-bc46-2848c31d9ce9-cert\") pod \"ingress-canary-prpjp\" (UID: \"5e0bcec3-73e6-4a2a-bc46-2848c31d9ce9\") " pod="openshift-ingress-canary/ingress-canary-prpjp" Jun 09 01:16:15.138824 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.138814 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4cca482-39bd-4a22-bed7-e85a83b32b47-tmp-dir\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:15.138975 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:15.138837 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 01:16:15.138975 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:15.138904 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls podName:c4cca482-39bd-4a22-bed7-e85a83b32b47 nodeName:}" failed. No retries permitted until 2026-06-09 01:16:15.638882311 +0000 UTC m=+36.657186481 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls") pod "dns-default-5vfr5" (UID: "c4cca482-39bd-4a22-bed7-e85a83b32b47") : secret "dns-default-metrics-tls" not found Jun 09 01:16:15.139076 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.138843 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c4cca482-39bd-4a22-bed7-e85a83b32b47-config-volume\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:15.139076 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.139041 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmsss\" (UniqueName: \"kubernetes.io/projected/5e0bcec3-73e6-4a2a-bc46-2848c31d9ce9-kube-api-access-dmsss\") pod \"ingress-canary-prpjp\" (UID: \"5e0bcec3-73e6-4a2a-bc46-2848c31d9ce9\") " pod="openshift-ingress-canary/ingress-canary-prpjp" Jun 09 01:16:15.139218 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.139147 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c4cca482-39bd-4a22-bed7-e85a83b32b47-tmp-dir\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:15.139376 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.139360 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c4cca482-39bd-4a22-bed7-e85a83b32b47-config-volume\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:15.143786 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.143745 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5e0bcec3-73e6-4a2a-bc46-2848c31d9ce9-cert\") pod \"ingress-canary-prpjp\" (UID: \"5e0bcec3-73e6-4a2a-bc46-2848c31d9ce9\") " pod="openshift-ingress-canary/ingress-canary-prpjp" Jun 09 01:16:15.147145 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.147126 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-269lb\" (UniqueName: \"kubernetes.io/projected/c4cca482-39bd-4a22-bed7-e85a83b32b47-kube-api-access-269lb\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:15.153898 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.153877 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmsss\" (UniqueName: \"kubernetes.io/projected/5e0bcec3-73e6-4a2a-bc46-2848c31d9ce9-kube-api-access-dmsss\") pod \"ingress-canary-prpjp\" (UID: \"5e0bcec3-73e6-4a2a-bc46-2848c31d9ce9\") " pod="openshift-ingress-canary/ingress-canary-prpjp" Jun 09 01:16:15.214779 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.214757 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-prpjp" Jun 09 01:16:15.384078 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.384050 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-prpjp"] Jun 09 01:16:15.387254 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:16:15.387230 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e0bcec3_73e6_4a2a_bc46_2848c31d9ce9.slice/crio-5902bd07983c1959e6abe95f03a70393d39419d5d4e8188d7c2cd9c8de50662c WatchSource:0}: Error finding container 5902bd07983c1959e6abe95f03a70393d39419d5d4e8188d7c2cd9c8de50662c: Status 404 returned error can't find the container with id 5902bd07983c1959e6abe95f03a70393d39419d5d4e8188d7c2cd9c8de50662c Jun 09 01:16:15.410201 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.410162 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:16:15.410597 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.410572 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:16:15.415424 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.415193 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 09 01:16:15.415424 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.415289 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 09 01:16:15.415424 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.415308 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ljzbq\"" Jun 09 01:16:15.415424 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.415407 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 09 01:16:15.415424 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.415420 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6pwvx\"" Jun 09 01:16:15.642407 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.642363 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:15.642513 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:15.642497 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 01:16:15.642582 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:15.642572 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls podName:c4cca482-39bd-4a22-bed7-e85a83b32b47 nodeName:}" failed. No retries permitted until 2026-06-09 01:16:16.642537263 +0000 UTC m=+37.660841433 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls") pod "dns-default-5vfr5" (UID: "c4cca482-39bd-4a22-bed7-e85a83b32b47") : secret "dns-default-metrics-tls" not found Jun 09 01:16:15.711967 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:15.711944 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-prpjp" event={"ID":"5e0bcec3-73e6-4a2a-bc46-2848c31d9ce9","Type":"ContainerStarted","Data":"5902bd07983c1959e6abe95f03a70393d39419d5d4e8188d7c2cd9c8de50662c"} Jun 09 01:16:16.409373 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:16.409342 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:16:16.412408 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:16.412382 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 09 01:16:16.648184 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:16.648019 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:16.648324 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:16.648166 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 01:16:16.648324 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:16.648310 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls podName:c4cca482-39bd-4a22-bed7-e85a83b32b47 nodeName:}" failed. No retries permitted until 2026-06-09 01:16:18.648291718 +0000 UTC m=+39.666595900 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls") pod "dns-default-5vfr5" (UID: "c4cca482-39bd-4a22-bed7-e85a83b32b47") : secret "dns-default-metrics-tls" not found Jun 09 01:16:18.663411 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:18.663377 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:18.663863 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:18.663537 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 01:16:18.663863 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:18.663613 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls podName:c4cca482-39bd-4a22-bed7-e85a83b32b47 nodeName:}" failed. No retries permitted until 2026-06-09 01:16:22.663595734 +0000 UTC m=+43.681899904 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls") pod "dns-default-5vfr5" (UID: "c4cca482-39bd-4a22-bed7-e85a83b32b47") : secret "dns-default-metrics-tls" not found Jun 09 01:16:19.722418 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:19.722387 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-prpjp" event={"ID":"5e0bcec3-73e6-4a2a-bc46-2848c31d9ce9","Type":"ContainerStarted","Data":"bd97b9d2a905eb702a807a8b5061ec7b91a3b95c7d1328d60b9db7708c725dce"} Jun 09 01:16:19.724880 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:19.724853 2555 generic.go:356] "Generic (PLEG): container finished" podID="fbd5c78f-a01c-41d6-81ce-fd01b8b78e43" containerID="9df03ed7f39e4ed0dcc2e1bd2f1c9ec0fb45391b84ec55bd7bfbaa14e8e10732" exitCode=0 Jun 09 01:16:19.724985 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:19.724906 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zrlt" event={"ID":"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43","Type":"ContainerDied","Data":"9df03ed7f39e4ed0dcc2e1bd2f1c9ec0fb45391b84ec55bd7bfbaa14e8e10732"} Jun 09 01:16:19.738419 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:19.738383 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-prpjp" podStartSLOduration=1.9518511680000001 podStartE2EDuration="5.738369891s" totalImagesPullingTime="3.786518723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:16:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:16:19.738092045 +0000 UTC m=+40.756396239" watchObservedRunningTime="2026-06-09 01:16:19.738369891 +0000 UTC m=+40.756674074" Jun 09 01:16:20.729999 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:20.729968 2555 generic.go:356] "Generic (PLEG): container finished" podID="fbd5c78f-a01c-41d6-81ce-fd01b8b78e43" containerID="2b85088999c726c1538ce6b299c6aaa07fff4780e3fdb00e83c2f1eaf04e1c41" exitCode=0 Jun 09 01:16:20.730344 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:20.730056 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zrlt" event={"ID":"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43","Type":"ContainerDied","Data":"2b85088999c726c1538ce6b299c6aaa07fff4780e3fdb00e83c2f1eaf04e1c41"} Jun 09 01:16:21.734832 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:21.734803 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5zrlt" event={"ID":"fbd5c78f-a01c-41d6-81ce-fd01b8b78e43","Type":"ContainerStarted","Data":"64ac7dd9492906eddccf380aab8b6fed2d217a8459e1be69d23dcc52666d3d29"} Jun 09 01:16:21.763202 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:21.763152 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5zrlt" podStartSLOduration=7.963353162 podStartE2EDuration="42.76313892s" totalImagesPullingTime="34.799785758s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 01:15:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:16:21.7611866 +0000 UTC m=+42.779490792" watchObservedRunningTime="2026-06-09 01:16:21.76313892 +0000 UTC m=+42.781443111" Jun 09 01:16:22.689299 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:22.689271 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:22.689436 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:22.689397 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 01:16:22.689474 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:22.689455 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls podName:c4cca482-39bd-4a22-bed7-e85a83b32b47 nodeName:}" failed. No retries permitted until 2026-06-09 01:16:30.689440846 +0000 UTC m=+51.707745017 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls") pod "dns-default-5vfr5" (UID: "c4cca482-39bd-4a22-bed7-e85a83b32b47") : secret "dns-default-metrics-tls" not found Jun 09 01:16:30.740923 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:30.740874 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:30.741338 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:30.741000 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 01:16:30.741338 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:30.741055 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls podName:c4cca482-39bd-4a22-bed7-e85a83b32b47 nodeName:}" failed. No retries permitted until 2026-06-09 01:16:46.741040609 +0000 UTC m=+67.759344779 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls") pod "dns-default-5vfr5" (UID: "c4cca482-39bd-4a22-bed7-e85a83b32b47") : secret "dns-default-metrics-tls" not found Jun 09 01:16:40.718162 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:40.718131 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gpkxb" Jun 09 01:16:44.130755 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:44.130696 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs\") pod \"network-metrics-daemon-8prh2\" (UID: \"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19\") " pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:16:44.133984 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:44.133963 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 09 01:16:44.141431 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:44.141412 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 09 01:16:44.141483 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:44.141474 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs podName:8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19 nodeName:}" failed. No retries permitted until 2026-06-09 01:17:48.141455434 +0000 UTC m=+129.159759604 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs") pod "network-metrics-daemon-8prh2" (UID: "8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19") : secret "metrics-daemon-secret" not found Jun 09 01:16:44.231350 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:44.231302 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret\") pod \"global-pull-secret-syncer-wgbm8\" (UID: \"76027187-053a-4879-b929-163968a1cd82\") " pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:16:44.231449 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:44.231375 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r62gc\" (UniqueName: \"kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc\") pod \"network-check-target-s4jjf\" (UID: \"42e3c300-dc21-44d9-a8a1-6cd16b11dc77\") " pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:16:44.234475 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:44.234455 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 09 01:16:44.234562 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:44.234530 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 09 01:16:44.244679 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:44.244650 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 09 01:16:44.245031 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:44.245011 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76027187-053a-4879-b929-163968a1cd82-original-pull-secret\") pod \"global-pull-secret-syncer-wgbm8\" (UID: \"76027187-053a-4879-b929-163968a1cd82\") " pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:16:44.254304 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:44.254279 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r62gc\" (UniqueName: \"kubernetes.io/projected/42e3c300-dc21-44d9-a8a1-6cd16b11dc77-kube-api-access-r62gc\") pod \"network-check-target-s4jjf\" (UID: \"42e3c300-dc21-44d9-a8a1-6cd16b11dc77\") " pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:16:44.319500 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:44.319441 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wgbm8" Jun 09 01:16:44.434661 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:44.434520 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wgbm8"] Jun 09 01:16:44.437001 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:16:44.436976 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76027187_053a_4879_b929_163968a1cd82.slice/crio-6e6eb793b60388d3cec1d78eb0e2d54c3fc3b813f162116cd1a8a7da60bd5c16 WatchSource:0}: Error finding container 6e6eb793b60388d3cec1d78eb0e2d54c3fc3b813f162116cd1a8a7da60bd5c16: Status 404 returned error can't find the container with id 6e6eb793b60388d3cec1d78eb0e2d54c3fc3b813f162116cd1a8a7da60bd5c16 Jun 09 01:16:44.533071 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:44.533039 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ljzbq\"" Jun 09 01:16:44.540512 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:44.540491 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:16:44.652963 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:44.652892 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-s4jjf"] Jun 09 01:16:44.655327 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:16:44.655298 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42e3c300_dc21_44d9_a8a1_6cd16b11dc77.slice/crio-118a6bd2f4476e4556b0124d27e0a016bcc971c4d205dc73a2408e0e792deeef WatchSource:0}: Error finding container 118a6bd2f4476e4556b0124d27e0a016bcc971c4d205dc73a2408e0e792deeef: Status 404 returned error can't find the container with id 118a6bd2f4476e4556b0124d27e0a016bcc971c4d205dc73a2408e0e792deeef Jun 09 01:16:44.788513 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:44.788478 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wgbm8" event={"ID":"76027187-053a-4879-b929-163968a1cd82","Type":"ContainerStarted","Data":"6e6eb793b60388d3cec1d78eb0e2d54c3fc3b813f162116cd1a8a7da60bd5c16"} Jun 09 01:16:44.789461 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:44.789440 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-s4jjf" event={"ID":"42e3c300-dc21-44d9-a8a1-6cd16b11dc77","Type":"ContainerStarted","Data":"118a6bd2f4476e4556b0124d27e0a016bcc971c4d205dc73a2408e0e792deeef"} Jun 09 01:16:46.747367 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:46.747333 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:16:46.747821 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:46.747488 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 01:16:46.747821 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:16:46.747575 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls podName:c4cca482-39bd-4a22-bed7-e85a83b32b47 nodeName:}" failed. No retries permitted until 2026-06-09 01:17:18.747537486 +0000 UTC m=+99.765841676 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls") pod "dns-default-5vfr5" (UID: "c4cca482-39bd-4a22-bed7-e85a83b32b47") : secret "dns-default-metrics-tls" not found Jun 09 01:16:49.804288 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:49.804245 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wgbm8" event={"ID":"76027187-053a-4879-b929-163968a1cd82","Type":"ContainerStarted","Data":"6e147b39ce93b47757d5fc9cb1fb20d885f25a68258af045977fbff5cdf13a1e"} Jun 09 01:16:49.805811 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:49.805781 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-s4jjf" event={"ID":"42e3c300-dc21-44d9-a8a1-6cd16b11dc77","Type":"ContainerStarted","Data":"6a79ee571669923372fa4b78fa998102eb1ba3bfb1779d456a6f89c71f8f8012"} Jun 09 01:16:49.805933 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:49.805885 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:16:49.819202 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:49.819163 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wgbm8" podStartSLOduration=65.847237664 podStartE2EDuration="1m10.819150053s" totalImagesPullingTime="4.971912389s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 01:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:16:49.818764297 +0000 UTC m=+70.837068485" watchObservedRunningTime="2026-06-09 01:16:49.819150053 +0000 UTC m=+70.837454249" Jun 09 01:16:49.833859 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:16:49.833818 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-s4jjf" podStartSLOduration=66.069245639 podStartE2EDuration="1m10.833806157s" totalImagesPullingTime="4.764560518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:16:49.832659453 +0000 UTC m=+70.850963645" watchObservedRunningTime="2026-06-09 01:16:49.833806157 +0000 UTC m=+70.852110348" Jun 09 01:17:18.749795 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:17:18.749749 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:17:18.750149 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:17:18.749902 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 01:17:18.750149 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:17:18.749981 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls podName:c4cca482-39bd-4a22-bed7-e85a83b32b47 nodeName:}" failed. No retries permitted until 2026-06-09 01:18:22.749961388 +0000 UTC m=+163.768265558 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls") pod "dns-default-5vfr5" (UID: "c4cca482-39bd-4a22-bed7-e85a83b32b47") : secret "dns-default-metrics-tls" not found Jun 09 01:17:20.810534 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:17:20.810506 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-s4jjf" Jun 09 01:17:48.231877 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:17:48.231832 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs\") pod \"network-metrics-daemon-8prh2\" (UID: \"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19\") " pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:17:48.232331 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:17:48.231948 2555 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 09 01:17:48.232331 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:17:48.232007 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs podName:8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19 nodeName:}" failed. No retries permitted until 2026-06-09 01:19:50.231991087 +0000 UTC m=+251.250295257 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs") pod "network-metrics-daemon-8prh2" (UID: "8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19") : secret "metrics-daemon-secret" not found Jun 09 01:18:17.891512 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:17.891425 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5vfr5" podUID="c4cca482-39bd-4a22-bed7-e85a83b32b47" Jun 09 01:18:18.007990 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:18.007960 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5vfr5" Jun 09 01:18:18.425970 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:18.425928 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8prh2" podUID="8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19" Jun 09 01:18:22.767199 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:22.767151 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:18:22.767708 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:22.767277 2555 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 01:18:22.767708 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:22.767369 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls podName:c4cca482-39bd-4a22-bed7-e85a83b32b47 nodeName:}" failed. No retries permitted until 2026-06-09 01:20:24.767345855 +0000 UTC m=+285.785650343 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls") pod "dns-default-5vfr5" (UID: "c4cca482-39bd-4a22-bed7-e85a83b32b47") : secret "dns-default-metrics-tls" not found Jun 09 01:18:29.411168 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:29.411132 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:18:35.721046 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.721006 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-shvgd"] Jun 09 01:18:35.724614 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.724590 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-shvgd" Jun 09 01:18:35.727849 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.727827 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b7949c658-4jkpz"] Jun 09 01:18:35.728901 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.728882 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jun 09 01:18:35.729236 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.729217 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-v6tsg\"" Jun 09 01:18:35.729335 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.729267 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jun 09 01:18:35.729556 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.729524 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.733392 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.733375 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 09 01:18:35.733392 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.733385 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 09 01:18:35.733976 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.733961 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 09 01:18:35.734296 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.734279 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8c589\"" Jun 09 01:18:35.738343 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.738270 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-shvgd"] Jun 09 01:18:35.739016 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.738994 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 09 01:18:35.748178 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.748161 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b7949c658-4jkpz"] Jun 09 01:18:35.818641 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.818603 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-ngl8x"] Jun 09 01:18:35.821073 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.820526 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-ngl8x" Jun 09 01:18:35.822436 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.822413 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4"] Jun 09 01:18:35.823841 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.823823 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-sd5t4\"" Jun 09 01:18:35.823999 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.823982 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-4rqfp"] Jun 09 01:18:35.824092 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.824077 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4" Jun 09 01:18:35.825765 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.825748 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx"] Jun 09 01:18:35.825882 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.825868 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:18:35.826829 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.826813 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 09 01:18:35.827030 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.827011 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 09 01:18:35.827030 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.827021 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pkqvq\"" Jun 09 01:18:35.827569 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.827538 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-896549976-jn4jq"] Jun 09 01:18:35.827640 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.827629 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx" Jun 09 01:18:35.829594 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.829577 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:35.831061 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.831037 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 09 01:18:35.831216 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.831202 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 09 01:18:35.831409 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.831393 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 09 01:18:35.831505 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.831487 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jun 09 01:18:35.831603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.831586 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-fff6f\"" Jun 09 01:18:35.831603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.831590 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-k4p4n\"" Jun 09 01:18:35.831719 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.831618 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jun 09 01:18:35.831719 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.831631 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jun 09 01:18:35.831855 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.831841 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jun 09 01:18:35.832147 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.832133 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 09 01:18:35.832360 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.832345 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jun 09 01:18:35.832417 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.832404 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 09 01:18:35.832650 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.832636 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 09 01:18:35.832876 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.832863 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jun 09 01:18:35.832914 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.832880 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-hst89\"" Jun 09 01:18:35.836672 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.836653 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-ngl8x"] Jun 09 01:18:35.838799 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.838779 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4"] Jun 09 01:18:35.839821 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.839800 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 09 01:18:35.841942 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.841923 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-4rqfp"] Jun 09 01:18:35.842415 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.842396 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jun 09 01:18:35.843188 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.843166 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-896549976-jn4jq"] Jun 09 01:18:35.844169 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.844148 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx"] Jun 09 01:18:35.856907 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.856884 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.857019 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.856922 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfk2g\" (UniqueName: \"kubernetes.io/projected/c564488d-eb77-4ed4-84a3-c3a465b41b4c-kube-api-access-pfk2g\") pod \"volume-data-source-validator-78dd696465-shvgd\" (UID: \"c564488d-eb77-4ed4-84a3-c3a465b41b4c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-shvgd" Jun 09 01:18:35.857019 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.856948 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-trusted-ca\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.857019 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.856993 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-bound-sa-token\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.857174 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.857048 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zshkq\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-kube-api-access-zshkq\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.857174 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.857094 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-ca-trust-extracted\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.857174 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.857149 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-certificates\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.857302 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.857175 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-installation-pull-secrets\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.857302 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.857203 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-image-registry-private-configuration\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.958313 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958278 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9464d89-5eba-4239-aff5-5f2430ecd991-tmp\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:35.958313 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958316 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zshkq\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-kube-api-access-zshkq\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.958522 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958337 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-ca-trust-extracted\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.958522 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958398 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9464d89-5eba-4239-aff5-5f2430ecd991-service-ca-bundle\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:35.958522 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958442 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-installation-pull-secrets\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.958522 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958474 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-image-registry-private-configuration\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.958522 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958503 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0855fbf3-db84-4b7c-b61a-c5dc9a63e530-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-f9xcx\" (UID: \"0855fbf3-db84-4b7c-b61a-c5dc9a63e530\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx" Jun 09 01:18:35.958743 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958535 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-s2qt4\" (UID: \"e247b35e-ed84-4ceb-ba24-eab0eebb8156\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4" Jun 09 01:18:35.958743 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958580 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hs5p\" (UniqueName: \"kubernetes.io/projected/0b11d16a-ae26-42b4-a44a-ac3b9d007579-kube-api-access-7hs5p\") pod \"console-operator-668fdbf85c-4rqfp\" (UID: \"0b11d16a-ae26-42b4-a44a-ac3b9d007579\") " pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:18:35.958743 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958604 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b11d16a-ae26-42b4-a44a-ac3b9d007579-serving-cert\") pod \"console-operator-668fdbf85c-4rqfp\" (UID: \"0b11d16a-ae26-42b4-a44a-ac3b9d007579\") " pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:18:35.958743 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958627 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frwzf\" (UniqueName: \"kubernetes.io/projected/4fbaec17-8c40-4a23-a4d9-b3db250e994b-kube-api-access-frwzf\") pod \"network-check-source-dbd9cf844-ngl8x\" (UID: \"4fbaec17-8c40-4a23-a4d9-b3db250e994b\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-ngl8x" Jun 09 01:18:35.958743 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958651 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0b11d16a-ae26-42b4-a44a-ac3b9d007579-trusted-ca\") pod \"console-operator-668fdbf85c-4rqfp\" (UID: \"0b11d16a-ae26-42b4-a44a-ac3b9d007579\") " pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:18:35.958743 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958687 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psq4h\" (UniqueName: \"kubernetes.io/projected/0855fbf3-db84-4b7c-b61a-c5dc9a63e530-kube-api-access-psq4h\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-f9xcx\" (UID: \"0855fbf3-db84-4b7c-b61a-c5dc9a63e530\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx" Jun 09 01:18:35.958743 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958723 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-ca-trust-extracted\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.958743 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958730 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-certificates\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.959028 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958750 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b11d16a-ae26-42b4-a44a-ac3b9d007579-config\") pod \"console-operator-668fdbf85c-4rqfp\" (UID: \"0b11d16a-ae26-42b4-a44a-ac3b9d007579\") " pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:18:35.959028 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958767 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj5sm\" (UniqueName: \"kubernetes.io/projected/b9464d89-5eba-4239-aff5-5f2430ecd991-kube-api-access-kj5sm\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:35.959028 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958786 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b9464d89-5eba-4239-aff5-5f2430ecd991-snapshots\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:35.959028 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958821 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e247b35e-ed84-4ceb-ba24-eab0eebb8156-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-s2qt4\" (UID: \"e247b35e-ed84-4ceb-ba24-eab0eebb8156\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4" Jun 09 01:18:35.959028 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958896 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0855fbf3-db84-4b7c-b61a-c5dc9a63e530-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-f9xcx\" (UID: \"0855fbf3-db84-4b7c-b61a-c5dc9a63e530\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx" Jun 09 01:18:35.959028 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958938 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.959028 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.958979 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfk2g\" (UniqueName: \"kubernetes.io/projected/c564488d-eb77-4ed4-84a3-c3a465b41b4c-kube-api-access-pfk2g\") pod \"volume-data-source-validator-78dd696465-shvgd\" (UID: \"c564488d-eb77-4ed4-84a3-c3a465b41b4c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-shvgd" Jun 09 01:18:35.959028 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.959006 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-bound-sa-token\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.959028 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:35.959012 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 01:18:35.959028 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:35.959029 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b7949c658-4jkpz: secret "image-registry-tls" not found Jun 09 01:18:35.959028 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.959032 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-trusted-ca\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.959436 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.959056 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9464d89-5eba-4239-aff5-5f2430ecd991-trusted-ca-bundle\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:35.959436 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:35.959085 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls podName:ebcc3a8a-5e34-41a0-82b8-8e1ba152608e nodeName:}" failed. No retries permitted until 2026-06-09 01:18:36.459066821 +0000 UTC m=+177.477371029 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls") pod "image-registry-6b7949c658-4jkpz" (UID: "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e") : secret "image-registry-tls" not found Jun 09 01:18:35.959436 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.959131 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9464d89-5eba-4239-aff5-5f2430ecd991-serving-cert\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:35.959436 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.959198 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-certificates\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.960031 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.960010 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-trusted-ca\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.960949 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.960929 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-installation-pull-secrets\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.961041 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.961018 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-image-registry-private-configuration\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.966191 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.966162 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zshkq\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-kube-api-access-zshkq\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:35.967030 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.967005 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfk2g\" (UniqueName: \"kubernetes.io/projected/c564488d-eb77-4ed4-84a3-c3a465b41b4c-kube-api-access-pfk2g\") pod \"volume-data-source-validator-78dd696465-shvgd\" (UID: \"c564488d-eb77-4ed4-84a3-c3a465b41b4c\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-shvgd" Jun 09 01:18:35.970217 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:35.970196 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-bound-sa-token\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:36.035403 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.035374 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-shvgd" Jun 09 01:18:36.060174 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.060143 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b9464d89-5eba-4239-aff5-5f2430ecd991-snapshots\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:36.060300 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.060190 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e247b35e-ed84-4ceb-ba24-eab0eebb8156-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-s2qt4\" (UID: \"e247b35e-ed84-4ceb-ba24-eab0eebb8156\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4" Jun 09 01:18:36.060300 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.060217 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0855fbf3-db84-4b7c-b61a-c5dc9a63e530-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-f9xcx\" (UID: \"0855fbf3-db84-4b7c-b61a-c5dc9a63e530\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx" Jun 09 01:18:36.060300 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.060266 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9464d89-5eba-4239-aff5-5f2430ecd991-trusted-ca-bundle\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:36.060300 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.060295 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9464d89-5eba-4239-aff5-5f2430ecd991-serving-cert\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:36.060488 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.060329 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9464d89-5eba-4239-aff5-5f2430ecd991-tmp\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:36.060565 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.060479 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9464d89-5eba-4239-aff5-5f2430ecd991-service-ca-bundle\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:36.061115 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.061088 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0855fbf3-db84-4b7c-b61a-c5dc9a63e530-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-f9xcx\" (UID: \"0855fbf3-db84-4b7c-b61a-c5dc9a63e530\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx" Jun 09 01:18:36.061227 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.060868 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b9464d89-5eba-4239-aff5-5f2430ecd991-snapshots\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:36.061227 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.061137 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-s2qt4\" (UID: \"e247b35e-ed84-4ceb-ba24-eab0eebb8156\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4" Jun 09 01:18:36.061227 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.061204 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hs5p\" (UniqueName: \"kubernetes.io/projected/0b11d16a-ae26-42b4-a44a-ac3b9d007579-kube-api-access-7hs5p\") pod \"console-operator-668fdbf85c-4rqfp\" (UID: \"0b11d16a-ae26-42b4-a44a-ac3b9d007579\") " pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:18:36.061382 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.061234 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b11d16a-ae26-42b4-a44a-ac3b9d007579-serving-cert\") pod \"console-operator-668fdbf85c-4rqfp\" (UID: \"0b11d16a-ae26-42b4-a44a-ac3b9d007579\") " pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:18:36.061382 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:36.061242 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 09 01:18:36.061382 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.061261 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frwzf\" (UniqueName: \"kubernetes.io/projected/4fbaec17-8c40-4a23-a4d9-b3db250e994b-kube-api-access-frwzf\") pod \"network-check-source-dbd9cf844-ngl8x\" (UID: \"4fbaec17-8c40-4a23-a4d9-b3db250e994b\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-ngl8x" Jun 09 01:18:36.061382 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.061291 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0b11d16a-ae26-42b4-a44a-ac3b9d007579-trusted-ca\") pod \"console-operator-668fdbf85c-4rqfp\" (UID: \"0b11d16a-ae26-42b4-a44a-ac3b9d007579\") " pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:18:36.061382 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:36.061314 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert podName:e247b35e-ed84-4ceb-ba24-eab0eebb8156 nodeName:}" failed. No retries permitted until 2026-06-09 01:18:36.561294885 +0000 UTC m=+177.579599071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-s2qt4" (UID: "e247b35e-ed84-4ceb-ba24-eab0eebb8156") : secret "networking-console-plugin-cert" not found Jun 09 01:18:36.061382 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.061317 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9464d89-5eba-4239-aff5-5f2430ecd991-trusted-ca-bundle\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:36.061382 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.061361 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9464d89-5eba-4239-aff5-5f2430ecd991-service-ca-bundle\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:36.061746 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.061383 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psq4h\" (UniqueName: \"kubernetes.io/projected/0855fbf3-db84-4b7c-b61a-c5dc9a63e530-kube-api-access-psq4h\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-f9xcx\" (UID: \"0855fbf3-db84-4b7c-b61a-c5dc9a63e530\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx" Jun 09 01:18:36.061746 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.061419 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b11d16a-ae26-42b4-a44a-ac3b9d007579-config\") pod \"console-operator-668fdbf85c-4rqfp\" (UID: \"0b11d16a-ae26-42b4-a44a-ac3b9d007579\") " pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:18:36.061746 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.061443 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj5sm\" (UniqueName: \"kubernetes.io/projected/b9464d89-5eba-4239-aff5-5f2430ecd991-kube-api-access-kj5sm\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:36.061890 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.061033 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e247b35e-ed84-4ceb-ba24-eab0eebb8156-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-s2qt4\" (UID: \"e247b35e-ed84-4ceb-ba24-eab0eebb8156\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4" Jun 09 01:18:36.061890 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.061027 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9464d89-5eba-4239-aff5-5f2430ecd991-tmp\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:36.061990 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.061906 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0855fbf3-db84-4b7c-b61a-c5dc9a63e530-config\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-f9xcx\" (UID: \"0855fbf3-db84-4b7c-b61a-c5dc9a63e530\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx" Jun 09 01:18:36.062322 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.062111 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0b11d16a-ae26-42b4-a44a-ac3b9d007579-trusted-ca\") pod \"console-operator-668fdbf85c-4rqfp\" (UID: \"0b11d16a-ae26-42b4-a44a-ac3b9d007579\") " pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:18:36.062322 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.062243 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b11d16a-ae26-42b4-a44a-ac3b9d007579-config\") pod \"console-operator-668fdbf85c-4rqfp\" (UID: \"0b11d16a-ae26-42b4-a44a-ac3b9d007579\") " pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:18:36.062787 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.062766 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0855fbf3-db84-4b7c-b61a-c5dc9a63e530-serving-cert\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-f9xcx\" (UID: \"0855fbf3-db84-4b7c-b61a-c5dc9a63e530\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx" Jun 09 01:18:36.063029 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.063010 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b9464d89-5eba-4239-aff5-5f2430ecd991-serving-cert\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:36.063858 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.063837 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b11d16a-ae26-42b4-a44a-ac3b9d007579-serving-cert\") pod \"console-operator-668fdbf85c-4rqfp\" (UID: \"0b11d16a-ae26-42b4-a44a-ac3b9d007579\") " pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:18:36.071332 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.071310 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psq4h\" (UniqueName: \"kubernetes.io/projected/0855fbf3-db84-4b7c-b61a-c5dc9a63e530-kube-api-access-psq4h\") pod \"kube-storage-version-migrator-operator-55bb7c4d49-f9xcx\" (UID: \"0855fbf3-db84-4b7c-b61a-c5dc9a63e530\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx" Jun 09 01:18:36.072084 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.072060 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hs5p\" (UniqueName: \"kubernetes.io/projected/0b11d16a-ae26-42b4-a44a-ac3b9d007579-kube-api-access-7hs5p\") pod \"console-operator-668fdbf85c-4rqfp\" (UID: \"0b11d16a-ae26-42b4-a44a-ac3b9d007579\") " pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:18:36.072790 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.072748 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj5sm\" (UniqueName: \"kubernetes.io/projected/b9464d89-5eba-4239-aff5-5f2430ecd991-kube-api-access-kj5sm\") pod \"insights-operator-896549976-jn4jq\" (UID: \"b9464d89-5eba-4239-aff5-5f2430ecd991\") " pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:36.072942 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.072920 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frwzf\" (UniqueName: \"kubernetes.io/projected/4fbaec17-8c40-4a23-a4d9-b3db250e994b-kube-api-access-frwzf\") pod \"network-check-source-dbd9cf844-ngl8x\" (UID: \"4fbaec17-8c40-4a23-a4d9-b3db250e994b\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-ngl8x" Jun 09 01:18:36.131964 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.131932 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-ngl8x" Jun 09 01:18:36.151928 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.151895 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:18:36.154105 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.153889 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-shvgd"] Jun 09 01:18:36.155820 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:18:36.155795 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc564488d_eb77_4ed4_84a3_c3a465b41b4c.slice/crio-0f4a69b6dbc54d4df79b0eaa5867622c35e266e93a6c2c608daf5c88d880564a WatchSource:0}: Error finding container 0f4a69b6dbc54d4df79b0eaa5867622c35e266e93a6c2c608daf5c88d880564a: Status 404 returned error can't find the container with id 0f4a69b6dbc54d4df79b0eaa5867622c35e266e93a6c2c608daf5c88d880564a Jun 09 01:18:36.157017 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.156991 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx" Jun 09 01:18:36.162872 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.162851 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-896549976-jn4jq" Jun 09 01:18:36.277529 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.277500 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-ngl8x"] Jun 09 01:18:36.280754 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:18:36.280721 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fbaec17_8c40_4a23_a4d9_b3db250e994b.slice/crio-8db21373a91932fec951ebe118857f5e604419a4d407c08338db21af3baae997 WatchSource:0}: Error finding container 8db21373a91932fec951ebe118857f5e604419a4d407c08338db21af3baae997: Status 404 returned error can't find the container with id 8db21373a91932fec951ebe118857f5e604419a4d407c08338db21af3baae997 Jun 09 01:18:36.332615 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.332587 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-896549976-jn4jq"] Jun 09 01:18:36.334791 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:18:36.334770 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9464d89_5eba_4239_aff5_5f2430ecd991.slice/crio-b1ecc271fe12aa4061a8ec75ce8c9e026e1d3a054e39ca33dd9d6b573d9d41a6 WatchSource:0}: Error finding container b1ecc271fe12aa4061a8ec75ce8c9e026e1d3a054e39ca33dd9d6b573d9d41a6: Status 404 returned error can't find the container with id b1ecc271fe12aa4061a8ec75ce8c9e026e1d3a054e39ca33dd9d6b573d9d41a6 Jun 09 01:18:36.464775 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.464738 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:36.465024 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:36.464909 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 01:18:36.465024 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:36.464929 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b7949c658-4jkpz: secret "image-registry-tls" not found Jun 09 01:18:36.465024 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:36.464983 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls podName:ebcc3a8a-5e34-41a0-82b8-8e1ba152608e nodeName:}" failed. No retries permitted until 2026-06-09 01:18:37.464968163 +0000 UTC m=+178.483272347 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls") pod "image-registry-6b7949c658-4jkpz" (UID: "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e") : secret "image-registry-tls" not found Jun 09 01:18:36.511673 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.511621 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx"] Jun 09 01:18:36.514502 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:18:36.514469 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0855fbf3_db84_4b7c_b61a_c5dc9a63e530.slice/crio-1e23289b171ce8c8d588a1810f4ca72c46cc918526e50a641be4fe56ec02abb3 WatchSource:0}: Error finding container 1e23289b171ce8c8d588a1810f4ca72c46cc918526e50a641be4fe56ec02abb3: Status 404 returned error can't find the container with id 1e23289b171ce8c8d588a1810f4ca72c46cc918526e50a641be4fe56ec02abb3 Jun 09 01:18:36.514625 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.514605 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-4rqfp"] Jun 09 01:18:36.517439 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:18:36.517418 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b11d16a_ae26_42b4_a44a_ac3b9d007579.slice/crio-edc2f86dd25d7e7d2bced169f6b6372b6396f4dd9791d12f7bcfefd1c918ec11 WatchSource:0}: Error finding container edc2f86dd25d7e7d2bced169f6b6372b6396f4dd9791d12f7bcfefd1c918ec11: Status 404 returned error can't find the container with id edc2f86dd25d7e7d2bced169f6b6372b6396f4dd9791d12f7bcfefd1c918ec11 Jun 09 01:18:36.565706 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:36.565622 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-s2qt4\" (UID: \"e247b35e-ed84-4ceb-ba24-eab0eebb8156\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4" Jun 09 01:18:36.565844 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:36.565801 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 09 01:18:36.565888 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:36.565868 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert podName:e247b35e-ed84-4ceb-ba24-eab0eebb8156 nodeName:}" failed. No retries permitted until 2026-06-09 01:18:37.565853242 +0000 UTC m=+178.584157412 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-s2qt4" (UID: "e247b35e-ed84-4ceb-ba24-eab0eebb8156") : secret "networking-console-plugin-cert" not found Jun 09 01:18:37.059136 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:37.059061 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx" event={"ID":"0855fbf3-db84-4b7c-b61a-c5dc9a63e530","Type":"ContainerStarted","Data":"1e23289b171ce8c8d588a1810f4ca72c46cc918526e50a641be4fe56ec02abb3"} Jun 09 01:18:37.060830 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:37.060777 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" event={"ID":"0b11d16a-ae26-42b4-a44a-ac3b9d007579","Type":"ContainerStarted","Data":"edc2f86dd25d7e7d2bced169f6b6372b6396f4dd9791d12f7bcfefd1c918ec11"} Jun 09 01:18:37.062346 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:37.062316 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-shvgd" event={"ID":"c564488d-eb77-4ed4-84a3-c3a465b41b4c","Type":"ContainerStarted","Data":"0f4a69b6dbc54d4df79b0eaa5867622c35e266e93a6c2c608daf5c88d880564a"} Jun 09 01:18:37.063524 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:37.063493 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-jn4jq" event={"ID":"b9464d89-5eba-4239-aff5-5f2430ecd991","Type":"ContainerStarted","Data":"b1ecc271fe12aa4061a8ec75ce8c9e026e1d3a054e39ca33dd9d6b573d9d41a6"} Jun 09 01:18:37.065107 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:37.065087 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-ngl8x" event={"ID":"4fbaec17-8c40-4a23-a4d9-b3db250e994b","Type":"ContainerStarted","Data":"c96a135578c94b1e4eeb6cc9f33c9e6cbc1e08ee7af4911e4429aa45cc2a0889"} Jun 09 01:18:37.065192 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:37.065123 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-ngl8x" event={"ID":"4fbaec17-8c40-4a23-a4d9-b3db250e994b","Type":"ContainerStarted","Data":"8db21373a91932fec951ebe118857f5e604419a4d407c08338db21af3baae997"} Jun 09 01:18:37.084126 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:37.084075 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-ngl8x" podStartSLOduration=2.084060924 podStartE2EDuration="2.084060924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:18:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:18:37.083070552 +0000 UTC m=+178.101374745" watchObservedRunningTime="2026-06-09 01:18:37.084060924 +0000 UTC m=+178.102365151" Jun 09 01:18:37.473181 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:37.473091 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:37.473419 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:37.473400 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 01:18:37.473481 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:37.473423 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b7949c658-4jkpz: secret "image-registry-tls" not found Jun 09 01:18:37.473537 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:37.473488 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls podName:ebcc3a8a-5e34-41a0-82b8-8e1ba152608e nodeName:}" failed. No retries permitted until 2026-06-09 01:18:39.473469027 +0000 UTC m=+180.491773198 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls") pod "image-registry-6b7949c658-4jkpz" (UID: "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e") : secret "image-registry-tls" not found Jun 09 01:18:37.574076 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:37.574036 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-s2qt4\" (UID: \"e247b35e-ed84-4ceb-ba24-eab0eebb8156\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4" Jun 09 01:18:37.574241 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:37.574207 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 09 01:18:37.574313 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:37.574274 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert podName:e247b35e-ed84-4ceb-ba24-eab0eebb8156 nodeName:}" failed. No retries permitted until 2026-06-09 01:18:39.574256586 +0000 UTC m=+180.592560778 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-s2qt4" (UID: "e247b35e-ed84-4ceb-ba24-eab0eebb8156") : secret "networking-console-plugin-cert" not found Jun 09 01:18:38.072947 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:38.072864 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-shvgd" event={"ID":"c564488d-eb77-4ed4-84a3-c3a465b41b4c","Type":"ContainerStarted","Data":"a3a2c435f8986c78c9441b529cdc9bb589c291642b6b95dd98553dca6a69fa62"} Jun 09 01:18:39.490364 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:39.490322 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:39.490856 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:39.490476 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 01:18:39.490856 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:39.490491 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b7949c658-4jkpz: secret "image-registry-tls" not found Jun 09 01:18:39.490856 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:39.490579 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls podName:ebcc3a8a-5e34-41a0-82b8-8e1ba152608e nodeName:}" failed. No retries permitted until 2026-06-09 01:18:43.490563587 +0000 UTC m=+184.508867774 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls") pod "image-registry-6b7949c658-4jkpz" (UID: "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e") : secret "image-registry-tls" not found Jun 09 01:18:39.590860 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:39.590825 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-s2qt4\" (UID: \"e247b35e-ed84-4ceb-ba24-eab0eebb8156\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4" Jun 09 01:18:39.590987 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:39.590968 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 09 01:18:39.591055 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:39.591046 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert podName:e247b35e-ed84-4ceb-ba24-eab0eebb8156 nodeName:}" failed. No retries permitted until 2026-06-09 01:18:43.591029185 +0000 UTC m=+184.609333355 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-s2qt4" (UID: "e247b35e-ed84-4ceb-ba24-eab0eebb8156") : secret "networking-console-plugin-cert" not found Jun 09 01:18:39.747654 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:39.747592 2555 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b11d16a_ae26_42b4_a44a_ac3b9d007579.slice/crio-conmon-12e264f61fc8549749224b62771ac545c38d738b57203ffd4df28b1199e44b2b.scope\": RecentStats: unable to find data in memory cache]" Jun 09 01:18:40.082254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:40.082215 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx" event={"ID":"0855fbf3-db84-4b7c-b61a-c5dc9a63e530","Type":"ContainerStarted","Data":"3864ac2d1329cdfd732a9567f7b71de180e53d3abccbbc15e6f61f299ebc6000"} Jun 09 01:18:40.083805 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:40.083785 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/0.log" Jun 09 01:18:40.083935 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:40.083820 2555 generic.go:356] "Generic (PLEG): container finished" podID="0b11d16a-ae26-42b4-a44a-ac3b9d007579" containerID="12e264f61fc8549749224b62771ac545c38d738b57203ffd4df28b1199e44b2b" exitCode=255 Jun 09 01:18:40.083935 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:40.083895 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" event={"ID":"0b11d16a-ae26-42b4-a44a-ac3b9d007579","Type":"ContainerDied","Data":"12e264f61fc8549749224b62771ac545c38d738b57203ffd4df28b1199e44b2b"} Jun 09 01:18:40.084127 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:40.084105 2555 scope.go:117] "RemoveContainer" containerID="12e264f61fc8549749224b62771ac545c38d738b57203ffd4df28b1199e44b2b" Jun 09 01:18:40.085388 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:40.085367 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-jn4jq" event={"ID":"b9464d89-5eba-4239-aff5-5f2430ecd991","Type":"ContainerStarted","Data":"be22a7b626166ddfcf89e6ebe1afd6147aff024c078ec100e9c15fe3988f6b32"} Jun 09 01:18:40.099582 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:40.099488 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx" podStartSLOduration=2.021636472 podStartE2EDuration="5.099471678s" totalImagesPullingTime="3.077835206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:18:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:18:40.097660181 +0000 UTC m=+181.115964372" watchObservedRunningTime="2026-06-09 01:18:40.099471678 +0000 UTC m=+181.117775867" Jun 09 01:18:40.100280 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:40.100249 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78dd696465-shvgd" podStartSLOduration=3.610105897 podStartE2EDuration="5.100235976s" totalImagesPullingTime="1.490130079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:18:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:18:38.088202143 +0000 UTC m=+179.106506340" watchObservedRunningTime="2026-06-09 01:18:40.100235976 +0000 UTC m=+181.118540168" Jun 09 01:18:40.115204 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:40.115152 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-896549976-jn4jq" podStartSLOduration=1.850739312 podStartE2EDuration="5.115135087s" totalImagesPullingTime="3.264395775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:18:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:18:40.114334098 +0000 UTC m=+181.132638290" watchObservedRunningTime="2026-06-09 01:18:40.115135087 +0000 UTC m=+181.133439281" Jun 09 01:18:41.090291 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:41.090259 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/1.log" Jun 09 01:18:41.090683 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:41.090631 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/0.log" Jun 09 01:18:41.090683 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:41.090666 2555 generic.go:356] "Generic (PLEG): container finished" podID="0b11d16a-ae26-42b4-a44a-ac3b9d007579" containerID="e3310b4a1c3462cbb2548e90471b8aa2b2dccccc998aca0ca975b5e25b66f633" exitCode=255 Jun 09 01:18:41.090787 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:41.090760 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" event={"ID":"0b11d16a-ae26-42b4-a44a-ac3b9d007579","Type":"ContainerDied","Data":"e3310b4a1c3462cbb2548e90471b8aa2b2dccccc998aca0ca975b5e25b66f633"} Jun 09 01:18:41.090872 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:41.090859 2555 scope.go:117] "RemoveContainer" containerID="12e264f61fc8549749224b62771ac545c38d738b57203ffd4df28b1199e44b2b" Jun 09 01:18:41.091120 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:41.091101 2555 scope.go:117] "RemoveContainer" containerID="e3310b4a1c3462cbb2548e90471b8aa2b2dccccc998aca0ca975b5e25b66f633" Jun 09 01:18:41.091310 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:41.091287 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-4rqfp_openshift-console-operator(0b11d16a-ae26-42b4-a44a-ac3b9d007579)\"" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" podUID="0b11d16a-ae26-42b4-a44a-ac3b9d007579" Jun 09 01:18:42.095436 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:42.095405 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/1.log" Jun 09 01:18:42.095911 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:42.095813 2555 scope.go:117] "RemoveContainer" containerID="e3310b4a1c3462cbb2548e90471b8aa2b2dccccc998aca0ca975b5e25b66f633" Jun 09 01:18:42.096042 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:42.096020 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-4rqfp_openshift-console-operator(0b11d16a-ae26-42b4-a44a-ac3b9d007579)\"" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" podUID="0b11d16a-ae26-42b4-a44a-ac3b9d007579" Jun 09 01:18:43.026850 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:43.026819 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rs95s_8e1cf77d-cef0-4363-a02f-f0887da4e984/dns-node-resolver/0.log" Jun 09 01:18:43.521494 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:43.521461 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:43.521887 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:43.521601 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 01:18:43.521887 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:43.521618 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b7949c658-4jkpz: secret "image-registry-tls" not found Jun 09 01:18:43.521887 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:43.521670 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls podName:ebcc3a8a-5e34-41a0-82b8-8e1ba152608e nodeName:}" failed. No retries permitted until 2026-06-09 01:18:51.52165494 +0000 UTC m=+192.539959110 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls") pod "image-registry-6b7949c658-4jkpz" (UID: "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e") : secret "image-registry-tls" not found Jun 09 01:18:43.622686 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:43.622646 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-s2qt4\" (UID: \"e247b35e-ed84-4ceb-ba24-eab0eebb8156\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4" Jun 09 01:18:43.622865 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:43.622804 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 09 01:18:43.622908 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:43.622869 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert podName:e247b35e-ed84-4ceb-ba24-eab0eebb8156 nodeName:}" failed. No retries permitted until 2026-06-09 01:18:51.622853844 +0000 UTC m=+192.641158036 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-s2qt4" (UID: "e247b35e-ed84-4ceb-ba24-eab0eebb8156") : secret "networking-console-plugin-cert" not found Jun 09 01:18:43.826473 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:43.826403 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-87mlr_4108bdff-ba06-42ab-9b64-a7be1cf9c8a4/node-ca/0.log" Jun 09 01:18:44.426107 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:44.426080 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-prpjp_5e0bcec3-73e6-4a2a-bc46-2848c31d9ce9/serve-healthcheck-canary/0.log" Jun 09 01:18:46.152621 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.152583 2555 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:18:46.153028 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.152664 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:18:46.153028 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.152972 2555 scope.go:117] "RemoveContainer" containerID="e3310b4a1c3462cbb2548e90471b8aa2b2dccccc998aca0ca975b5e25b66f633" Jun 09 01:18:46.153129 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:46.153108 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-4rqfp_openshift-console-operator(0b11d16a-ae26-42b4-a44a-ac3b9d007579)\"" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" podUID="0b11d16a-ae26-42b4-a44a-ac3b9d007579" Jun 09 01:18:46.281188 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.281155 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-l94gz"] Jun 09 01:18:46.284118 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.284104 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-l94gz" Jun 09 01:18:46.286907 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.286885 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 09 01:18:46.287044 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.286940 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 09 01:18:46.288324 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.288304 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-qvcm2\"" Jun 09 01:18:46.288419 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.288354 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 09 01:18:46.288419 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.288364 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 09 01:18:46.290822 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.290774 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-l94gz"] Jun 09 01:18:46.443129 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.443032 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e91ab802-c33f-4ca4-a7ac-2eba96d6a305-signing-key\") pod \"service-ca-7d74d578b8-l94gz\" (UID: \"e91ab802-c33f-4ca4-a7ac-2eba96d6a305\") " pod="openshift-service-ca/service-ca-7d74d578b8-l94gz" Jun 09 01:18:46.443129 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.443086 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x7t9\" (UniqueName: \"kubernetes.io/projected/e91ab802-c33f-4ca4-a7ac-2eba96d6a305-kube-api-access-9x7t9\") pod \"service-ca-7d74d578b8-l94gz\" (UID: \"e91ab802-c33f-4ca4-a7ac-2eba96d6a305\") " pod="openshift-service-ca/service-ca-7d74d578b8-l94gz" Jun 09 01:18:46.443129 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.443117 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e91ab802-c33f-4ca4-a7ac-2eba96d6a305-signing-cabundle\") pod \"service-ca-7d74d578b8-l94gz\" (UID: \"e91ab802-c33f-4ca4-a7ac-2eba96d6a305\") " pod="openshift-service-ca/service-ca-7d74d578b8-l94gz" Jun 09 01:18:46.544384 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.544340 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e91ab802-c33f-4ca4-a7ac-2eba96d6a305-signing-key\") pod \"service-ca-7d74d578b8-l94gz\" (UID: \"e91ab802-c33f-4ca4-a7ac-2eba96d6a305\") " pod="openshift-service-ca/service-ca-7d74d578b8-l94gz" Jun 09 01:18:46.544489 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.544401 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x7t9\" (UniqueName: \"kubernetes.io/projected/e91ab802-c33f-4ca4-a7ac-2eba96d6a305-kube-api-access-9x7t9\") pod \"service-ca-7d74d578b8-l94gz\" (UID: \"e91ab802-c33f-4ca4-a7ac-2eba96d6a305\") " pod="openshift-service-ca/service-ca-7d74d578b8-l94gz" Jun 09 01:18:46.544489 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.544423 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e91ab802-c33f-4ca4-a7ac-2eba96d6a305-signing-cabundle\") pod \"service-ca-7d74d578b8-l94gz\" (UID: \"e91ab802-c33f-4ca4-a7ac-2eba96d6a305\") " pod="openshift-service-ca/service-ca-7d74d578b8-l94gz" Jun 09 01:18:46.545137 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.545114 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e91ab802-c33f-4ca4-a7ac-2eba96d6a305-signing-cabundle\") pod \"service-ca-7d74d578b8-l94gz\" (UID: \"e91ab802-c33f-4ca4-a7ac-2eba96d6a305\") " pod="openshift-service-ca/service-ca-7d74d578b8-l94gz" Jun 09 01:18:46.546738 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.546719 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e91ab802-c33f-4ca4-a7ac-2eba96d6a305-signing-key\") pod \"service-ca-7d74d578b8-l94gz\" (UID: \"e91ab802-c33f-4ca4-a7ac-2eba96d6a305\") " pod="openshift-service-ca/service-ca-7d74d578b8-l94gz" Jun 09 01:18:46.552442 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.552413 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x7t9\" (UniqueName: \"kubernetes.io/projected/e91ab802-c33f-4ca4-a7ac-2eba96d6a305-kube-api-access-9x7t9\") pod \"service-ca-7d74d578b8-l94gz\" (UID: \"e91ab802-c33f-4ca4-a7ac-2eba96d6a305\") " pod="openshift-service-ca/service-ca-7d74d578b8-l94gz" Jun 09 01:18:46.592224 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.592193 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-l94gz" Jun 09 01:18:46.712009 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:46.711936 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-l94gz"] Jun 09 01:18:46.714693 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:18:46.714664 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode91ab802_c33f_4ca4_a7ac_2eba96d6a305.slice/crio-12eff02099a16c9b3bf6523e72ca8c7b1bc0429a6c44cdbac7ac4080c3ff3151 WatchSource:0}: Error finding container 12eff02099a16c9b3bf6523e72ca8c7b1bc0429a6c44cdbac7ac4080c3ff3151: Status 404 returned error can't find the container with id 12eff02099a16c9b3bf6523e72ca8c7b1bc0429a6c44cdbac7ac4080c3ff3151 Jun 09 01:18:47.112342 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:47.112306 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-l94gz" event={"ID":"e91ab802-c33f-4ca4-a7ac-2eba96d6a305","Type":"ContainerStarted","Data":"12eff02099a16c9b3bf6523e72ca8c7b1bc0429a6c44cdbac7ac4080c3ff3151"} Jun 09 01:18:49.120038 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:49.119999 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-l94gz" event={"ID":"e91ab802-c33f-4ca4-a7ac-2eba96d6a305","Type":"ContainerStarted","Data":"8c0d5bd6fd119e7e364d3c6c9d7fa76c44e8fad99dd8958ee9573b340f9858b2"} Jun 09 01:18:49.137692 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:49.137639 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7d74d578b8-l94gz" podStartSLOduration=1.1629767229999999 podStartE2EDuration="3.137621777s" totalImagesPullingTime="1.974645054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:18:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:18:49.135758377 +0000 UTC m=+190.154062568" watchObservedRunningTime="2026-06-09 01:18:49.137621777 +0000 UTC m=+190.155925969" Jun 09 01:18:51.590005 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:51.589957 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:18:51.590495 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:51.590144 2555 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 01:18:51.590495 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:51.590172 2555 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b7949c658-4jkpz: secret "image-registry-tls" not found Jun 09 01:18:51.590495 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:51.590262 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls podName:ebcc3a8a-5e34-41a0-82b8-8e1ba152608e nodeName:}" failed. No retries permitted until 2026-06-09 01:19:07.590238272 +0000 UTC m=+208.608542455 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls") pod "image-registry-6b7949c658-4jkpz" (UID: "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e") : secret "image-registry-tls" not found Jun 09 01:18:51.691382 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:51.691350 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-s2qt4\" (UID: \"e247b35e-ed84-4ceb-ba24-eab0eebb8156\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4" Jun 09 01:18:51.691531 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:51.691468 2555 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 09 01:18:51.691531 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:18:51.691520 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert podName:e247b35e-ed84-4ceb-ba24-eab0eebb8156 nodeName:}" failed. No retries permitted until 2026-06-09 01:19:07.691507088 +0000 UTC m=+208.709811258 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-s2qt4" (UID: "e247b35e-ed84-4ceb-ba24-eab0eebb8156") : secret "networking-console-plugin-cert" not found Jun 09 01:18:59.412252 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:18:59.412224 2555 scope.go:117] "RemoveContainer" containerID="e3310b4a1c3462cbb2548e90471b8aa2b2dccccc998aca0ca975b5e25b66f633" Jun 09 01:19:00.153640 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:00.153614 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:19:00.154018 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:00.154003 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/1.log" Jun 09 01:19:00.154064 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:00.154037 2555 generic.go:356] "Generic (PLEG): container finished" podID="0b11d16a-ae26-42b4-a44a-ac3b9d007579" containerID="927264b3301bb501635b803b239609fddc69026bc4743ad7f4b9a11a2ad57536" exitCode=255 Jun 09 01:19:00.154108 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:00.154092 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" event={"ID":"0b11d16a-ae26-42b4-a44a-ac3b9d007579","Type":"ContainerDied","Data":"927264b3301bb501635b803b239609fddc69026bc4743ad7f4b9a11a2ad57536"} Jun 09 01:19:00.154155 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:00.154146 2555 scope.go:117] "RemoveContainer" containerID="e3310b4a1c3462cbb2548e90471b8aa2b2dccccc998aca0ca975b5e25b66f633" Jun 09 01:19:00.154500 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:00.154484 2555 scope.go:117] "RemoveContainer" containerID="927264b3301bb501635b803b239609fddc69026bc4743ad7f4b9a11a2ad57536" Jun 09 01:19:00.154676 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:19:00.154657 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-668fdbf85c-4rqfp_openshift-console-operator(0b11d16a-ae26-42b4-a44a-ac3b9d007579)\"" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" podUID="0b11d16a-ae26-42b4-a44a-ac3b9d007579" Jun 09 01:19:01.157962 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:01.157937 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:19:06.152466 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.152431 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:19:06.152831 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.152484 2555 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:19:06.152831 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.152809 2555 scope.go:117] "RemoveContainer" containerID="927264b3301bb501635b803b239609fddc69026bc4743ad7f4b9a11a2ad57536" Jun 09 01:19:06.152973 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:19:06.152951 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-668fdbf85c-4rqfp_openshift-console-operator(0b11d16a-ae26-42b4-a44a-ac3b9d007579)\"" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" podUID="0b11d16a-ae26-42b4-a44a-ac3b9d007579" Jun 09 01:19:06.537148 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.537113 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b7949c658-4jkpz"] Jun 09 01:19:06.537338 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:19:06.537318 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" podUID="ebcc3a8a-5e34-41a0-82b8-8e1ba152608e" Jun 09 01:19:06.546909 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.546882 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qwn6v"] Jun 09 01:19:06.549748 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.549731 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qwn6v" Jun 09 01:19:06.552486 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.552457 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 09 01:19:06.552608 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.552565 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 09 01:19:06.552881 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.552866 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-knzt7\"" Jun 09 01:19:06.562873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.562856 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qwn6v"] Jun 09 01:19:06.609383 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.609346 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b96de010-76d0-4a70-9437-d2d54dfa7631-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qwn6v\" (UID: \"b96de010-76d0-4a70-9437-d2d54dfa7631\") " pod="openshift-insights/insights-runtime-extractor-qwn6v" Jun 09 01:19:06.609618 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.609415 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b96de010-76d0-4a70-9437-d2d54dfa7631-data-volume\") pod \"insights-runtime-extractor-qwn6v\" (UID: \"b96de010-76d0-4a70-9437-d2d54dfa7631\") " pod="openshift-insights/insights-runtime-extractor-qwn6v" Jun 09 01:19:06.609618 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.609474 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b96de010-76d0-4a70-9437-d2d54dfa7631-crio-socket\") pod \"insights-runtime-extractor-qwn6v\" (UID: \"b96de010-76d0-4a70-9437-d2d54dfa7631\") " pod="openshift-insights/insights-runtime-extractor-qwn6v" Jun 09 01:19:06.609618 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.609531 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq62g\" (UniqueName: \"kubernetes.io/projected/b96de010-76d0-4a70-9437-d2d54dfa7631-kube-api-access-dq62g\") pod \"insights-runtime-extractor-qwn6v\" (UID: \"b96de010-76d0-4a70-9437-d2d54dfa7631\") " pod="openshift-insights/insights-runtime-extractor-qwn6v" Jun 09 01:19:06.609618 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.609590 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b96de010-76d0-4a70-9437-d2d54dfa7631-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qwn6v\" (UID: \"b96de010-76d0-4a70-9437-d2d54dfa7631\") " pod="openshift-insights/insights-runtime-extractor-qwn6v" Jun 09 01:19:06.633272 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.633236 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd"] Jun 09 01:19:06.635395 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.635373 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.648698 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.648672 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd"] Jun 09 01:19:06.710757 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.710722 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b96de010-76d0-4a70-9437-d2d54dfa7631-crio-socket\") pod \"insights-runtime-extractor-qwn6v\" (UID: \"b96de010-76d0-4a70-9437-d2d54dfa7631\") " pod="openshift-insights/insights-runtime-extractor-qwn6v" Jun 09 01:19:06.710942 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.710762 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e52b9d7c-548a-4cf5-933c-7e2e0969e797-ca-trust-extracted\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.710942 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.710808 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq62g\" (UniqueName: \"kubernetes.io/projected/b96de010-76d0-4a70-9437-d2d54dfa7631-kube-api-access-dq62g\") pod \"insights-runtime-extractor-qwn6v\" (UID: \"b96de010-76d0-4a70-9437-d2d54dfa7631\") " pod="openshift-insights/insights-runtime-extractor-qwn6v" Jun 09 01:19:06.710942 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.710837 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b96de010-76d0-4a70-9437-d2d54dfa7631-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qwn6v\" (UID: \"b96de010-76d0-4a70-9437-d2d54dfa7631\") " pod="openshift-insights/insights-runtime-extractor-qwn6v" Jun 09 01:19:06.710942 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.710848 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b96de010-76d0-4a70-9437-d2d54dfa7631-crio-socket\") pod \"insights-runtime-extractor-qwn6v\" (UID: \"b96de010-76d0-4a70-9437-d2d54dfa7631\") " pod="openshift-insights/insights-runtime-extractor-qwn6v" Jun 09 01:19:06.710942 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.710867 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e52b9d7c-548a-4cf5-933c-7e2e0969e797-registry-certificates\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.710942 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.710892 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e52b9d7c-548a-4cf5-933c-7e2e0969e797-trusted-ca\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.711217 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.710985 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e52b9d7c-548a-4cf5-933c-7e2e0969e797-registry-tls\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.711217 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.711012 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e52b9d7c-548a-4cf5-933c-7e2e0969e797-installation-pull-secrets\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.711217 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.711048 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b96de010-76d0-4a70-9437-d2d54dfa7631-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qwn6v\" (UID: \"b96de010-76d0-4a70-9437-d2d54dfa7631\") " pod="openshift-insights/insights-runtime-extractor-qwn6v" Jun 09 01:19:06.711217 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.711082 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e52b9d7c-548a-4cf5-933c-7e2e0969e797-bound-sa-token\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.711217 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.711121 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e52b9d7c-548a-4cf5-933c-7e2e0969e797-image-registry-private-configuration\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.711217 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.711150 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw45f\" (UniqueName: \"kubernetes.io/projected/e52b9d7c-548a-4cf5-933c-7e2e0969e797-kube-api-access-hw45f\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.711217 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.711196 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b96de010-76d0-4a70-9437-d2d54dfa7631-data-volume\") pod \"insights-runtime-extractor-qwn6v\" (UID: \"b96de010-76d0-4a70-9437-d2d54dfa7631\") " pod="openshift-insights/insights-runtime-extractor-qwn6v" Jun 09 01:19:06.711514 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.711493 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b96de010-76d0-4a70-9437-d2d54dfa7631-data-volume\") pod \"insights-runtime-extractor-qwn6v\" (UID: \"b96de010-76d0-4a70-9437-d2d54dfa7631\") " pod="openshift-insights/insights-runtime-extractor-qwn6v" Jun 09 01:19:06.711568 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.711558 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b96de010-76d0-4a70-9437-d2d54dfa7631-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qwn6v\" (UID: \"b96de010-76d0-4a70-9437-d2d54dfa7631\") " pod="openshift-insights/insights-runtime-extractor-qwn6v" Jun 09 01:19:06.713053 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.713029 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b96de010-76d0-4a70-9437-d2d54dfa7631-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qwn6v\" (UID: \"b96de010-76d0-4a70-9437-d2d54dfa7631\") " pod="openshift-insights/insights-runtime-extractor-qwn6v" Jun 09 01:19:06.721094 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.721073 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq62g\" (UniqueName: \"kubernetes.io/projected/b96de010-76d0-4a70-9437-d2d54dfa7631-kube-api-access-dq62g\") pod \"insights-runtime-extractor-qwn6v\" (UID: \"b96de010-76d0-4a70-9437-d2d54dfa7631\") " pod="openshift-insights/insights-runtime-extractor-qwn6v" Jun 09 01:19:06.811604 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.811518 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e52b9d7c-548a-4cf5-933c-7e2e0969e797-registry-tls\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.811604 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.811577 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e52b9d7c-548a-4cf5-933c-7e2e0969e797-installation-pull-secrets\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.811604 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.811607 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e52b9d7c-548a-4cf5-933c-7e2e0969e797-bound-sa-token\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.811871 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.811626 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e52b9d7c-548a-4cf5-933c-7e2e0969e797-image-registry-private-configuration\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.811871 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.811641 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw45f\" (UniqueName: \"kubernetes.io/projected/e52b9d7c-548a-4cf5-933c-7e2e0969e797-kube-api-access-hw45f\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.811871 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.811673 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e52b9d7c-548a-4cf5-933c-7e2e0969e797-ca-trust-extracted\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.811871 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.811724 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e52b9d7c-548a-4cf5-933c-7e2e0969e797-registry-certificates\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.811871 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.811748 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e52b9d7c-548a-4cf5-933c-7e2e0969e797-trusted-ca\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.812161 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.812137 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e52b9d7c-548a-4cf5-933c-7e2e0969e797-ca-trust-extracted\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.812748 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.812724 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e52b9d7c-548a-4cf5-933c-7e2e0969e797-registry-certificates\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.812850 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.812835 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e52b9d7c-548a-4cf5-933c-7e2e0969e797-trusted-ca\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.814129 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.814106 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e52b9d7c-548a-4cf5-933c-7e2e0969e797-installation-pull-secrets\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.814443 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.814424 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e52b9d7c-548a-4cf5-933c-7e2e0969e797-image-registry-private-configuration\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.814485 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.814433 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e52b9d7c-548a-4cf5-933c-7e2e0969e797-registry-tls\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.819497 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.819472 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw45f\" (UniqueName: \"kubernetes.io/projected/e52b9d7c-548a-4cf5-933c-7e2e0969e797-kube-api-access-hw45f\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.819883 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.819865 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e52b9d7c-548a-4cf5-933c-7e2e0969e797-bound-sa-token\") pod \"image-registry-5f9bcfd4d8-m4rzd\" (UID: \"e52b9d7c-548a-4cf5-933c-7e2e0969e797\") " pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.858512 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.858483 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qwn6v" Jun 09 01:19:06.946837 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.946811 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8c589\"" Jun 09 01:19:06.954213 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.954185 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:06.980185 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:06.980159 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qwn6v"] Jun 09 01:19:06.982590 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:19:06.982561 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb96de010_76d0_4a70_9437_d2d54dfa7631.slice/crio-5811db32b43946272b9617a6d0f4debec4c5433508d3ce0230111533021a1acf WatchSource:0}: Error finding container 5811db32b43946272b9617a6d0f4debec4c5433508d3ce0230111533021a1acf: Status 404 returned error can't find the container with id 5811db32b43946272b9617a6d0f4debec4c5433508d3ce0230111533021a1acf Jun 09 01:19:07.080187 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.080109 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd"] Jun 09 01:19:07.082913 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:19:07.082890 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode52b9d7c_548a_4cf5_933c_7e2e0969e797.slice/crio-7e147735d69894c2c19682c8867d69c6ecca05a33f417f42892141415f90c9b9 WatchSource:0}: Error finding container 7e147735d69894c2c19682c8867d69c6ecca05a33f417f42892141415f90c9b9: Status 404 returned error can't find the container with id 7e147735d69894c2c19682c8867d69c6ecca05a33f417f42892141415f90c9b9 Jun 09 01:19:07.176508 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.176457 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qwn6v" event={"ID":"b96de010-76d0-4a70-9437-d2d54dfa7631","Type":"ContainerStarted","Data":"251503d8dd80330a799561ef82cfd2037f90a3a972fd7807674c54f7150b9e57"} Jun 09 01:19:07.176972 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.176530 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qwn6v" event={"ID":"b96de010-76d0-4a70-9437-d2d54dfa7631","Type":"ContainerStarted","Data":"5811db32b43946272b9617a6d0f4debec4c5433508d3ce0230111533021a1acf"} Jun 09 01:19:07.178054 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.178025 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" event={"ID":"e52b9d7c-548a-4cf5-933c-7e2e0969e797","Type":"ContainerStarted","Data":"f1c3e91aa29df2df2b3ef3896bfb0d14be7f75412e1c10bec00e227adf88eb9b"} Jun 09 01:19:07.178054 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.178045 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:19:07.178224 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.178065 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" event={"ID":"e52b9d7c-548a-4cf5-933c-7e2e0969e797","Type":"ContainerStarted","Data":"7e147735d69894c2c19682c8867d69c6ecca05a33f417f42892141415f90c9b9"} Jun 09 01:19:07.183341 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.183321 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:19:07.214236 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.214195 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-certificates\") pod \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " Jun 09 01:19:07.214415 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.214273 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-ca-trust-extracted\") pod \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " Jun 09 01:19:07.214415 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.214293 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-image-registry-private-configuration\") pod \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " Jun 09 01:19:07.214415 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.214332 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-trusted-ca\") pod \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " Jun 09 01:19:07.214415 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.214361 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-installation-pull-secrets\") pod \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " Jun 09 01:19:07.214415 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.214391 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-bound-sa-token\") pod \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " Jun 09 01:19:07.214680 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.214423 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zshkq\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-kube-api-access-zshkq\") pod \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " Jun 09 01:19:07.214680 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.214639 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e" (UID: "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:19:07.214680 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.214639 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e" (UID: "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 01:19:07.215434 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.215117 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e" (UID: "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 01:19:07.215434 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.215359 2555 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-ca-trust-extracted\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:19:07.215434 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.215382 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-trusted-ca\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:19:07.215434 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.215398 2555 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-certificates\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:19:07.216747 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.216728 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e" (UID: "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:19:07.216890 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.216868 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e" (UID: "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 01:19:07.216890 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.216880 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e" (UID: "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 01:19:07.217428 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.217413 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-kube-api-access-zshkq" (OuterVolumeSpecName: "kube-api-access-zshkq") pod "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e" (UID: "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e"). InnerVolumeSpecName "kube-api-access-zshkq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:19:07.221521 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.221468 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" podStartSLOduration=1.221448799 podStartE2EDuration="1.221448799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:19:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:19:07.219859475 +0000 UTC m=+208.238163667" watchObservedRunningTime="2026-06-09 01:19:07.221448799 +0000 UTC m=+208.239752992" Jun 09 01:19:07.316797 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.316752 2555 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-image-registry-private-configuration\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:19:07.316797 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.316790 2555 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-installation-pull-secrets\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:19:07.316797 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.316803 2555 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-bound-sa-token\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:19:07.317024 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.316816 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zshkq\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-kube-api-access-zshkq\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:19:07.618247 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.618206 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:19:07.621556 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.621515 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls\") pod \"image-registry-6b7949c658-4jkpz\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:19:07.719666 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.719525 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls\") pod \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\" (UID: \"ebcc3a8a-5e34-41a0-82b8-8e1ba152608e\") " Jun 09 01:19:07.719888 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.719863 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-s2qt4\" (UID: \"e247b35e-ed84-4ceb-ba24-eab0eebb8156\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4" Jun 09 01:19:07.722004 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.721972 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e" (UID: "ebcc3a8a-5e34-41a0-82b8-8e1ba152608e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:19:07.722648 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.722469 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e247b35e-ed84-4ceb-ba24-eab0eebb8156-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-s2qt4\" (UID: \"e247b35e-ed84-4ceb-ba24-eab0eebb8156\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4" Jun 09 01:19:07.820756 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.820732 2555 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e-registry-tls\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:19:07.941390 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.941353 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-pkqvq\"" Jun 09 01:19:07.949575 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:07.949539 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4" Jun 09 01:19:08.067681 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:08.067648 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4"] Jun 09 01:19:08.070896 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:19:08.070871 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode247b35e_ed84_4ceb_ba24_eab0eebb8156.slice/crio-1ad98ca7ba238c5c072f94e86bb5370e1f58b72956a4efe490f1fd180fa32126 WatchSource:0}: Error finding container 1ad98ca7ba238c5c072f94e86bb5370e1f58b72956a4efe490f1fd180fa32126: Status 404 returned error can't find the container with id 1ad98ca7ba238c5c072f94e86bb5370e1f58b72956a4efe490f1fd180fa32126 Jun 09 01:19:08.183089 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:08.183050 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qwn6v" event={"ID":"b96de010-76d0-4a70-9437-d2d54dfa7631","Type":"ContainerStarted","Data":"cc93042528795adebde1332fdf19e0a120dfdd721a9626b5c76bed2a31f65174"} Jun 09 01:19:08.184139 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:08.184106 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4" event={"ID":"e247b35e-ed84-4ceb-ba24-eab0eebb8156","Type":"ContainerStarted","Data":"1ad98ca7ba238c5c072f94e86bb5370e1f58b72956a4efe490f1fd180fa32126"} Jun 09 01:19:08.184139 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:08.184130 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b7949c658-4jkpz" Jun 09 01:19:08.184556 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:08.184527 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:08.212741 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:08.212713 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b7949c658-4jkpz"] Jun 09 01:19:08.215980 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:08.215957 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6b7949c658-4jkpz"] Jun 09 01:19:09.414717 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:09.414524 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebcc3a8a-5e34-41a0-82b8-8e1ba152608e" path="/var/lib/kubelet/pods/ebcc3a8a-5e34-41a0-82b8-8e1ba152608e/volumes" Jun 09 01:19:10.192888 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:10.192856 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qwn6v" event={"ID":"b96de010-76d0-4a70-9437-d2d54dfa7631","Type":"ContainerStarted","Data":"6287ccc2ec4275d59eb7d7db7b82d192e8cf7ab9e9592a4599e99a0fe0a93e9f"} Jun 09 01:19:10.194265 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:10.194238 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4" event={"ID":"e247b35e-ed84-4ceb-ba24-eab0eebb8156","Type":"ContainerStarted","Data":"b11aa7dce8c30611dd1dbd910dcde3fc3ef37dd854a3b24f3d603bf243401d57"} Jun 09 01:19:10.217080 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:10.217029 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qwn6v" podStartSLOduration=1.673662109 podStartE2EDuration="4.217014846s" totalImagesPullingTime="2.543352737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 01:19:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:19:10.216257357 +0000 UTC m=+211.234561549" watchObservedRunningTime="2026-06-09 01:19:10.217014846 +0000 UTC m=+211.235319039" Jun 09 01:19:10.233927 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:10.233882 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-s2qt4" podStartSLOduration=33.668732827 podStartE2EDuration="35.233863371s" totalImagesPullingTime="1.565130544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:18:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:19:10.233019404 +0000 UTC m=+211.251323596" watchObservedRunningTime="2026-06-09 01:19:10.233863371 +0000 UTC m=+211.252167565" Jun 09 01:19:13.461832 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.461787 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2"] Jun 09 01:19:13.464906 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.464888 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" Jun 09 01:19:13.467573 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.467529 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 09 01:19:13.467573 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.467556 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 09 01:19:13.469140 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.469117 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-mxhgd\"" Jun 09 01:19:13.469271 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.469159 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 09 01:19:13.469271 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.469218 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 09 01:19:13.469402 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.469281 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 09 01:19:13.473379 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.473355 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2"] Jun 09 01:19:13.566271 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.566217 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e96446f7-1c13-4aec-9de2-c6ee863cfd66-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-m9ds2\" (UID: \"e96446f7-1c13-4aec-9de2-c6ee863cfd66\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" Jun 09 01:19:13.566452 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.566298 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8ppc\" (UniqueName: \"kubernetes.io/projected/e96446f7-1c13-4aec-9de2-c6ee863cfd66-kube-api-access-n8ppc\") pod \"prometheus-operator-7f7d445d84-m9ds2\" (UID: \"e96446f7-1c13-4aec-9de2-c6ee863cfd66\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" Jun 09 01:19:13.566452 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.566373 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e96446f7-1c13-4aec-9de2-c6ee863cfd66-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-m9ds2\" (UID: \"e96446f7-1c13-4aec-9de2-c6ee863cfd66\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" Jun 09 01:19:13.566452 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.566414 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e96446f7-1c13-4aec-9de2-c6ee863cfd66-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-m9ds2\" (UID: \"e96446f7-1c13-4aec-9de2-c6ee863cfd66\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" Jun 09 01:19:13.667083 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.667050 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e96446f7-1c13-4aec-9de2-c6ee863cfd66-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-m9ds2\" (UID: \"e96446f7-1c13-4aec-9de2-c6ee863cfd66\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" Jun 09 01:19:13.667247 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.667098 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8ppc\" (UniqueName: \"kubernetes.io/projected/e96446f7-1c13-4aec-9de2-c6ee863cfd66-kube-api-access-n8ppc\") pod \"prometheus-operator-7f7d445d84-m9ds2\" (UID: \"e96446f7-1c13-4aec-9de2-c6ee863cfd66\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" Jun 09 01:19:13.667247 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.667145 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e96446f7-1c13-4aec-9de2-c6ee863cfd66-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-m9ds2\" (UID: \"e96446f7-1c13-4aec-9de2-c6ee863cfd66\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" Jun 09 01:19:13.667247 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.667173 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e96446f7-1c13-4aec-9de2-c6ee863cfd66-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-m9ds2\" (UID: \"e96446f7-1c13-4aec-9de2-c6ee863cfd66\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" Jun 09 01:19:13.667900 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.667875 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e96446f7-1c13-4aec-9de2-c6ee863cfd66-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-m9ds2\" (UID: \"e96446f7-1c13-4aec-9de2-c6ee863cfd66\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" Jun 09 01:19:13.669524 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.669491 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e96446f7-1c13-4aec-9de2-c6ee863cfd66-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-m9ds2\" (UID: \"e96446f7-1c13-4aec-9de2-c6ee863cfd66\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" Jun 09 01:19:13.669656 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.669634 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e96446f7-1c13-4aec-9de2-c6ee863cfd66-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-m9ds2\" (UID: \"e96446f7-1c13-4aec-9de2-c6ee863cfd66\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" Jun 09 01:19:13.674854 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.674832 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8ppc\" (UniqueName: \"kubernetes.io/projected/e96446f7-1c13-4aec-9de2-c6ee863cfd66-kube-api-access-n8ppc\") pod \"prometheus-operator-7f7d445d84-m9ds2\" (UID: \"e96446f7-1c13-4aec-9de2-c6ee863cfd66\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" Jun 09 01:19:13.774959 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.774931 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" Jun 09 01:19:13.893556 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:13.893517 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2"] Jun 09 01:19:13.895962 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:19:13.895935 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode96446f7_1c13_4aec_9de2_c6ee863cfd66.slice/crio-2ec7a9d233671d4cfcca8117da9e7fe6ea772f39722c67243b68b9de59d0fe9c WatchSource:0}: Error finding container 2ec7a9d233671d4cfcca8117da9e7fe6ea772f39722c67243b68b9de59d0fe9c: Status 404 returned error can't find the container with id 2ec7a9d233671d4cfcca8117da9e7fe6ea772f39722c67243b68b9de59d0fe9c Jun 09 01:19:14.207470 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:14.207378 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" event={"ID":"e96446f7-1c13-4aec-9de2-c6ee863cfd66","Type":"ContainerStarted","Data":"2ec7a9d233671d4cfcca8117da9e7fe6ea772f39722c67243b68b9de59d0fe9c"} Jun 09 01:19:16.215553 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:16.215501 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" event={"ID":"e96446f7-1c13-4aec-9de2-c6ee863cfd66","Type":"ContainerStarted","Data":"02ade5f957101d9bac784cccf53ada73507b3a09e9c041b5f89ab96fedaab57c"} Jun 09 01:19:16.215937 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:16.215573 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" event={"ID":"e96446f7-1c13-4aec-9de2-c6ee863cfd66","Type":"ContainerStarted","Data":"46af5db97c4c30cc074e2aec84ee08c35299b298ae57e401e9acf3d2d6902534"} Jun 09 01:19:16.233198 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:16.233151 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7f7d445d84-m9ds2" podStartSLOduration=1.8027652939999999 podStartE2EDuration="3.233136076s" totalImagesPullingTime="1.430370782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:19:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:19:16.231360601 +0000 UTC m=+217.249664805" watchObservedRunningTime="2026-06-09 01:19:16.233136076 +0000 UTC m=+217.251440267" Jun 09 01:19:17.786138 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.786103 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7"] Jun 09 01:19:17.789194 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.789177 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" Jun 09 01:19:17.791900 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.791878 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-wddgg\"" Jun 09 01:19:17.792035 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.791878 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 09 01:19:17.792035 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.791877 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 09 01:19:17.799000 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.798980 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7"] Jun 09 01:19:17.810738 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.810716 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv"] Jun 09 01:19:17.813852 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.813827 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:17.814735 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.814715 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4fqcv"] Jun 09 01:19:17.816880 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.816857 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 09 01:19:17.817005 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.816869 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 09 01:19:17.817674 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.817659 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:17.818098 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.818077 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-vh9gp\"" Jun 09 01:19:17.818098 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.818090 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 09 01:19:17.820365 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.820346 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 09 01:19:17.820616 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.820599 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 09 01:19:17.820761 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.820644 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ngzk6\"" Jun 09 01:19:17.820837 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.820655 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 09 01:19:17.826028 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.826007 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv"] Jun 09 01:19:17.900091 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900055 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68646a62-0c47-413d-a54c-da26b53f5903-node-exporter-tls\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:17.900091 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900092 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68646a62-0c47-413d-a54c-da26b53f5903-root\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:17.900331 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900112 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68646a62-0c47-413d-a54c-da26b53f5903-sys\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:17.900331 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900139 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkfdk\" (UniqueName: \"kubernetes.io/projected/a17c6536-86cf-4097-87b7-7f7d8f864a36-kube-api-access-wkfdk\") pod \"openshift-state-metrics-65f78d5c66-dp8v7\" (UID: \"a17c6536-86cf-4097-87b7-7f7d8f864a36\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" Jun 09 01:19:17.900331 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900175 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:17.900331 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900198 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:17.900331 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900224 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/68646a62-0c47-413d-a54c-da26b53f5903-node-exporter-accelerators-collector-config\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:17.900331 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900254 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:17.900331 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900273 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:17.900331 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900288 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:17.900331 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900315 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68646a62-0c47-413d-a54c-da26b53f5903-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:17.900768 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900338 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/68646a62-0c47-413d-a54c-da26b53f5903-node-exporter-textfile\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:17.900768 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900404 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a17c6536-86cf-4097-87b7-7f7d8f864a36-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-dp8v7\" (UID: \"a17c6536-86cf-4097-87b7-7f7d8f864a36\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" Jun 09 01:19:17.900768 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900421 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68646a62-0c47-413d-a54c-da26b53f5903-node-exporter-wtmp\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:17.900768 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900439 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54zfr\" (UniqueName: \"kubernetes.io/projected/68646a62-0c47-413d-a54c-da26b53f5903-kube-api-access-54zfr\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:17.900768 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900460 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djx6n\" (UniqueName: \"kubernetes.io/projected/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-kube-api-access-djx6n\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:17.900768 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900574 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a17c6536-86cf-4097-87b7-7f7d8f864a36-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-dp8v7\" (UID: \"a17c6536-86cf-4097-87b7-7f7d8f864a36\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" Jun 09 01:19:17.900768 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900622 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a17c6536-86cf-4097-87b7-7f7d8f864a36-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-dp8v7\" (UID: \"a17c6536-86cf-4097-87b7-7f7d8f864a36\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" Jun 09 01:19:17.900768 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:17.900659 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68646a62-0c47-413d-a54c-da26b53f5903-metrics-client-ca\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.001393 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.001359 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/68646a62-0c47-413d-a54c-da26b53f5903-node-exporter-textfile\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.001612 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.001446 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a17c6536-86cf-4097-87b7-7f7d8f864a36-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-dp8v7\" (UID: \"a17c6536-86cf-4097-87b7-7f7d8f864a36\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" Jun 09 01:19:18.001612 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.001478 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68646a62-0c47-413d-a54c-da26b53f5903-node-exporter-wtmp\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.001612 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.001504 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54zfr\" (UniqueName: \"kubernetes.io/projected/68646a62-0c47-413d-a54c-da26b53f5903-kube-api-access-54zfr\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.001612 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.001533 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djx6n\" (UniqueName: \"kubernetes.io/projected/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-kube-api-access-djx6n\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:18.001612 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.001580 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a17c6536-86cf-4097-87b7-7f7d8f864a36-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-dp8v7\" (UID: \"a17c6536-86cf-4097-87b7-7f7d8f864a36\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" Jun 09 01:19:18.001612 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.001608 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a17c6536-86cf-4097-87b7-7f7d8f864a36-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-dp8v7\" (UID: \"a17c6536-86cf-4097-87b7-7f7d8f864a36\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" Jun 09 01:19:18.001914 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.001634 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68646a62-0c47-413d-a54c-da26b53f5903-metrics-client-ca\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.001914 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.001668 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68646a62-0c47-413d-a54c-da26b53f5903-node-exporter-tls\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.001914 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.001701 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68646a62-0c47-413d-a54c-da26b53f5903-root\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.001914 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.001729 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68646a62-0c47-413d-a54c-da26b53f5903-sys\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.001914 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:19:18.001756 2555 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 09 01:19:18.001914 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.001767 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/68646a62-0c47-413d-a54c-da26b53f5903-node-exporter-textfile\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.001914 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:19:18.001819 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a17c6536-86cf-4097-87b7-7f7d8f864a36-openshift-state-metrics-tls podName:a17c6536-86cf-4097-87b7-7f7d8f864a36 nodeName:}" failed. No retries permitted until 2026-06-09 01:19:18.501801635 +0000 UTC m=+219.520105811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a17c6536-86cf-4097-87b7-7f7d8f864a36-openshift-state-metrics-tls") pod "openshift-state-metrics-65f78d5c66-dp8v7" (UID: "a17c6536-86cf-4097-87b7-7f7d8f864a36") : secret "openshift-state-metrics-tls" not found Jun 09 01:19:18.001914 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.001757 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkfdk\" (UniqueName: \"kubernetes.io/projected/a17c6536-86cf-4097-87b7-7f7d8f864a36-kube-api-access-wkfdk\") pod \"openshift-state-metrics-65f78d5c66-dp8v7\" (UID: \"a17c6536-86cf-4097-87b7-7f7d8f864a36\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" Jun 09 01:19:18.002282 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.001936 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:18.002282 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.001966 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:18.002282 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.002003 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/68646a62-0c47-413d-a54c-da26b53f5903-node-exporter-accelerators-collector-config\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.002282 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.001665 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/68646a62-0c47-413d-a54c-da26b53f5903-node-exporter-wtmp\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.002282 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.002032 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:18.002282 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.002082 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:18.002282 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.002113 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:18.002282 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.002177 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68646a62-0c47-413d-a54c-da26b53f5903-sys\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.002282 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.002219 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a17c6536-86cf-4097-87b7-7f7d8f864a36-metrics-client-ca\") pod \"openshift-state-metrics-65f78d5c66-dp8v7\" (UID: \"a17c6536-86cf-4097-87b7-7f7d8f864a36\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" Jun 09 01:19:18.002282 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.002219 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68646a62-0c47-413d-a54c-da26b53f5903-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.002728 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.002705 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:18.003190 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.002871 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/68646a62-0c47-413d-a54c-da26b53f5903-root\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.003190 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.003011 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:18.003190 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.003012 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:18.003407 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.003208 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/68646a62-0c47-413d-a54c-da26b53f5903-node-exporter-accelerators-collector-config\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.003508 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.003482 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68646a62-0c47-413d-a54c-da26b53f5903-metrics-client-ca\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.005039 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.004907 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/68646a62-0c47-413d-a54c-da26b53f5903-node-exporter-tls\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.005039 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.004954 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:18.005244 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.005124 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a17c6536-86cf-4097-87b7-7f7d8f864a36-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-65f78d5c66-dp8v7\" (UID: \"a17c6536-86cf-4097-87b7-7f7d8f864a36\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" Jun 09 01:19:18.006112 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.006083 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:18.006112 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.006097 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68646a62-0c47-413d-a54c-da26b53f5903-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.012741 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.012696 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54zfr\" (UniqueName: \"kubernetes.io/projected/68646a62-0c47-413d-a54c-da26b53f5903-kube-api-access-54zfr\") pod \"node-exporter-4fqcv\" (UID: \"68646a62-0c47-413d-a54c-da26b53f5903\") " pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.012741 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.012714 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djx6n\" (UniqueName: \"kubernetes.io/projected/39d900f3-ce61-429d-a6a0-0662a6c8b6fd-kube-api-access-djx6n\") pod \"kube-state-metrics-57bbf8bfb5-5sgdv\" (UID: \"39d900f3-ce61-429d-a6a0-0662a6c8b6fd\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:18.012895 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.012714 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkfdk\" (UniqueName: \"kubernetes.io/projected/a17c6536-86cf-4097-87b7-7f7d8f864a36-kube-api-access-wkfdk\") pod \"openshift-state-metrics-65f78d5c66-dp8v7\" (UID: \"a17c6536-86cf-4097-87b7-7f7d8f864a36\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" Jun 09 01:19:18.126099 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.126006 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" Jun 09 01:19:18.132305 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.132281 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4fqcv" Jun 09 01:19:18.144066 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:19:18.144037 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68646a62_0c47_413d_a54c_da26b53f5903.slice/crio-bb83956337063e8e555bd204e2eee7e348b5f204222469a41f8ab24d1fc9fd3d WatchSource:0}: Error finding container bb83956337063e8e555bd204e2eee7e348b5f204222469a41f8ab24d1fc9fd3d: Status 404 returned error can't find the container with id bb83956337063e8e555bd204e2eee7e348b5f204222469a41f8ab24d1fc9fd3d Jun 09 01:19:18.224527 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.224487 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4fqcv" event={"ID":"68646a62-0c47-413d-a54c-da26b53f5903","Type":"ContainerStarted","Data":"bb83956337063e8e555bd204e2eee7e348b5f204222469a41f8ab24d1fc9fd3d"} Jun 09 01:19:18.263662 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.263624 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv"] Jun 09 01:19:18.266735 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:19:18.266703 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39d900f3_ce61_429d_a6a0_0662a6c8b6fd.slice/crio-4caf49fc2ed6bfef275e5f2c91baf659b2112228167f2f1e88b0378ef439cc1f WatchSource:0}: Error finding container 4caf49fc2ed6bfef275e5f2c91baf659b2112228167f2f1e88b0378ef439cc1f: Status 404 returned error can't find the container with id 4caf49fc2ed6bfef275e5f2c91baf659b2112228167f2f1e88b0378ef439cc1f Jun 09 01:19:18.505923 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.505838 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a17c6536-86cf-4097-87b7-7f7d8f864a36-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-dp8v7\" (UID: \"a17c6536-86cf-4097-87b7-7f7d8f864a36\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" Jun 09 01:19:18.508235 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.508213 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a17c6536-86cf-4097-87b7-7f7d8f864a36-openshift-state-metrics-tls\") pod \"openshift-state-metrics-65f78d5c66-dp8v7\" (UID: \"a17c6536-86cf-4097-87b7-7f7d8f864a36\") " pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" Jun 09 01:19:18.699628 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.699594 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" Jun 09 01:19:18.867205 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.867171 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7"] Jun 09 01:19:18.871198 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.871146 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 01:19:18.876706 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.876685 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:18.880971 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.879768 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 09 01:19:18.880971 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.880268 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 09 01:19:18.880971 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.880473 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 09 01:19:18.880971 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.880707 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-cl768\"" Jun 09 01:19:18.880971 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.880793 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 09 01:19:18.880971 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.880820 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 09 01:19:18.880971 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.880851 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 09 01:19:18.881384 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.880875 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 09 01:19:18.881384 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.880948 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 09 01:19:18.881714 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.881527 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 09 01:19:18.892171 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.891876 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 01:19:18.910057 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.909948 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f83e1756-56ca-42fb-a339-de1f84b4bec6-config-out\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:18.910057 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.910016 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f83e1756-56ca-42fb-a339-de1f84b4bec6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:18.910057 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.910043 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:18.910283 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.910069 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:18.910283 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.910089 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sl58\" (UniqueName: \"kubernetes.io/projected/f83e1756-56ca-42fb-a339-de1f84b4bec6-kube-api-access-7sl58\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:18.910283 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.910246 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:18.910517 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.910283 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:18.910517 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.910314 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-config-volume\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:18.910517 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.910348 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f83e1756-56ca-42fb-a339-de1f84b4bec6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:18.910517 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.910416 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f83e1756-56ca-42fb-a339-de1f84b4bec6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:18.910517 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.910469 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:18.910763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.910522 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-web-config\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:18.910763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:18.910597 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f83e1756-56ca-42fb-a339-de1f84b4bec6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.011690 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.011113 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.011690 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.011161 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.011690 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.011201 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-config-volume\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.011690 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.011229 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f83e1756-56ca-42fb-a339-de1f84b4bec6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.011690 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.011260 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f83e1756-56ca-42fb-a339-de1f84b4bec6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.011690 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:19:19.011290 2555 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jun 09 01:19:19.011690 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.011306 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.011690 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.011340 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-web-config\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.011690 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:19:19.011370 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-main-tls podName:f83e1756-56ca-42fb-a339-de1f84b4bec6 nodeName:}" failed. No retries permitted until 2026-06-09 01:19:19.51134826 +0000 UTC m=+220.529652448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "f83e1756-56ca-42fb-a339-de1f84b4bec6") : secret "alertmanager-main-tls" not found Jun 09 01:19:19.011690 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.011446 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f83e1756-56ca-42fb-a339-de1f84b4bec6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.011690 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.011488 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f83e1756-56ca-42fb-a339-de1f84b4bec6-config-out\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.011690 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.011584 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f83e1756-56ca-42fb-a339-de1f84b4bec6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.011690 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.011611 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.011690 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.011650 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.011690 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.011675 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sl58\" (UniqueName: \"kubernetes.io/projected/f83e1756-56ca-42fb-a339-de1f84b4bec6-kube-api-access-7sl58\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.011690 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:19:19.011691 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f83e1756-56ca-42fb-a339-de1f84b4bec6-alertmanager-trusted-ca-bundle podName:f83e1756-56ca-42fb-a339-de1f84b4bec6 nodeName:}" failed. No retries permitted until 2026-06-09 01:19:19.511670572 +0000 UTC m=+220.529974747 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/f83e1756-56ca-42fb-a339-de1f84b4bec6-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "f83e1756-56ca-42fb-a339-de1f84b4bec6") : configmap references non-existent config key: ca-bundle.crt Jun 09 01:19:19.012583 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.012292 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f83e1756-56ca-42fb-a339-de1f84b4bec6-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.012658 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.012596 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f83e1756-56ca-42fb-a339-de1f84b4bec6-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.015634 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.015418 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.016313 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.016272 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.017066 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.016705 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.018355 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.017906 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-config-volume\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.019431 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.018953 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.019431 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.019395 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-web-config\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.019951 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.019879 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f83e1756-56ca-42fb-a339-de1f84b4bec6-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.020299 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.020259 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f83e1756-56ca-42fb-a339-de1f84b4bec6-config-out\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.022328 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.022290 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sl58\" (UniqueName: \"kubernetes.io/projected/f83e1756-56ca-42fb-a339-de1f84b4bec6-kube-api-access-7sl58\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.230944 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.230907 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" event={"ID":"a17c6536-86cf-4097-87b7-7f7d8f864a36","Type":"ContainerStarted","Data":"eea641a750c746fefbcde669425ee0e15dd9b0d226f7d782b34d0669f9d79764"} Jun 09 01:19:19.231065 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.231014 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" event={"ID":"a17c6536-86cf-4097-87b7-7f7d8f864a36","Type":"ContainerStarted","Data":"c8c49f674cf23e46a8c39c83556c56a099c298b481ad1bc7566e8149222893c5"} Jun 09 01:19:19.232669 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.232634 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" event={"ID":"39d900f3-ce61-429d-a6a0-0662a6c8b6fd","Type":"ContainerStarted","Data":"4caf49fc2ed6bfef275e5f2c91baf659b2112228167f2f1e88b0378ef439cc1f"} Jun 09 01:19:19.411897 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.411866 2555 scope.go:117] "RemoveContainer" containerID="927264b3301bb501635b803b239609fddc69026bc4743ad7f4b9a11a2ad57536" Jun 09 01:19:19.412093 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:19:19.412068 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-668fdbf85c-4rqfp_openshift-console-operator(0b11d16a-ae26-42b4-a44a-ac3b9d007579)\"" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" podUID="0b11d16a-ae26-42b4-a44a-ac3b9d007579" Jun 09 01:19:19.516229 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.516148 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.516229 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.516196 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f83e1756-56ca-42fb-a339-de1f84b4bec6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.517214 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.517187 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f83e1756-56ca-42fb-a339-de1f84b4bec6-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.519168 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.519144 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.794835 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.794814 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:19:19.960810 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:19.960707 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 01:19:19.963102 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:19:19.963057 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf83e1756_56ca_42fb_a339_de1f84b4bec6.slice/crio-a8634dfa7380749915063896dec15ccc8fb2966cf84810d72ffa5acf5c0cd67d WatchSource:0}: Error finding container a8634dfa7380749915063896dec15ccc8fb2966cf84810d72ffa5acf5c0cd67d: Status 404 returned error can't find the container with id a8634dfa7380749915063896dec15ccc8fb2966cf84810d72ffa5acf5c0cd67d Jun 09 01:19:20.238512 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:20.238421 2555 generic.go:356] "Generic (PLEG): container finished" podID="68646a62-0c47-413d-a54c-da26b53f5903" containerID="2e8852e164450940c62af005343b86f64bfcaf26973ed7ed5196efb1e17565aa" exitCode=0 Jun 09 01:19:20.238742 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:20.238509 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4fqcv" event={"ID":"68646a62-0c47-413d-a54c-da26b53f5903","Type":"ContainerDied","Data":"2e8852e164450940c62af005343b86f64bfcaf26973ed7ed5196efb1e17565aa"} Jun 09 01:19:20.241190 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:20.240913 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" event={"ID":"a17c6536-86cf-4097-87b7-7f7d8f864a36","Type":"ContainerStarted","Data":"ebc31c554f1bd4e5176745b667164b6eb3190668c4e1ba4caa3cdbc15b945bc2"} Jun 09 01:19:20.243413 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:20.243382 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" event={"ID":"39d900f3-ce61-429d-a6a0-0662a6c8b6fd","Type":"ContainerStarted","Data":"cfe316a2c4483229d55b60e80411827253c3469a4b1243502f52cccbf1031876"} Jun 09 01:19:20.243529 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:20.243440 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" event={"ID":"39d900f3-ce61-429d-a6a0-0662a6c8b6fd","Type":"ContainerStarted","Data":"0d157f4c32276d5191061c407c7532cb3411f9a119b54b896e1a80db454f33de"} Jun 09 01:19:20.243529 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:20.243470 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" event={"ID":"39d900f3-ce61-429d-a6a0-0662a6c8b6fd","Type":"ContainerStarted","Data":"749f41cdd2f1ed6e07cefb19b83184bf310183870b8ab16fcc4f84132e8465b2"} Jun 09 01:19:20.244680 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:20.244656 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f83e1756-56ca-42fb-a339-de1f84b4bec6","Type":"ContainerStarted","Data":"a8634dfa7380749915063896dec15ccc8fb2966cf84810d72ffa5acf5c0cd67d"} Jun 09 01:19:20.275470 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:20.275418 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-5sgdv" podStartSLOduration=1.879513313 podStartE2EDuration="3.275405058s" totalImagesPullingTime="1.395891745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:19:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:19:20.274828578 +0000 UTC m=+221.293132783" watchObservedRunningTime="2026-06-09 01:19:20.275405058 +0000 UTC m=+221.293709228" Jun 09 01:19:21.249593 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:21.249530 2555 generic.go:356] "Generic (PLEG): container finished" podID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerID="eee142e3a00cc6ac81bb73bd24a52cce112443e7d1ae37a09b1ce18858ade79c" exitCode=0 Jun 09 01:19:21.250008 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:21.249613 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f83e1756-56ca-42fb-a339-de1f84b4bec6","Type":"ContainerDied","Data":"eee142e3a00cc6ac81bb73bd24a52cce112443e7d1ae37a09b1ce18858ade79c"} Jun 09 01:19:21.251842 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:21.251818 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4fqcv" event={"ID":"68646a62-0c47-413d-a54c-da26b53f5903","Type":"ContainerStarted","Data":"301f15ba7e6e6819a017baed2357842c583355462db15e8f3121db74eef7c424"} Jun 09 01:19:21.251943 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:21.251866 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4fqcv" event={"ID":"68646a62-0c47-413d-a54c-da26b53f5903","Type":"ContainerStarted","Data":"4bc8e3d4fbbc7d27e194fa7f3364646f4d2cbdccdf6304b8e3d0b01b3d7407e4"} Jun 09 01:19:21.253824 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:21.253804 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" event={"ID":"a17c6536-86cf-4097-87b7-7f7d8f864a36","Type":"ContainerStarted","Data":"7a4fc52cc611ee4d4438e70efa8372975a032e96276748c93063be5891a23b63"} Jun 09 01:19:21.291366 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:21.291320 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-65f78d5c66-dp8v7" podStartSLOduration=3.197379713 podStartE2EDuration="4.291305569s" totalImagesPullingTime="1.093925856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:19:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:19:21.290902993 +0000 UTC m=+222.309207185" watchObservedRunningTime="2026-06-09 01:19:21.291305569 +0000 UTC m=+222.309609761" Jun 09 01:19:21.309832 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:21.309766 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4fqcv" podStartSLOduration=3.140968258 podStartE2EDuration="4.309752659s" totalImagesPullingTime="1.100033327s" totalInitContainerRuntime="68.751074ms" isStatefulPod=true podCreationTimestamp="2026-06-09 01:19:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:19:21.309395876 +0000 UTC m=+222.327700090" watchObservedRunningTime="2026-06-09 01:19:21.309752659 +0000 UTC m=+222.328056852" Jun 09 01:19:23.263276 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:23.263250 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f83e1756-56ca-42fb-a339-de1f84b4bec6","Type":"ContainerStarted","Data":"52059020fcd736fce03d6469c4168cab3d674c728f863e3ca73fd24a0207bc52"} Jun 09 01:19:23.263574 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:23.263313 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f83e1756-56ca-42fb-a339-de1f84b4bec6","Type":"ContainerStarted","Data":"25c271d4b275334da919fc0508358519bc8bd714209a2044290c5699139af98e"} Jun 09 01:19:24.271497 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:24.271461 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f83e1756-56ca-42fb-a339-de1f84b4bec6","Type":"ContainerStarted","Data":"30ca63b6b6e9dcacd1162b593006407f9aec73808776f0c393e0366bdae8d350"} Jun 09 01:19:24.271979 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:24.271526 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f83e1756-56ca-42fb-a339-de1f84b4bec6","Type":"ContainerStarted","Data":"a2781225e75705ca684cd104a93626a191fdae7a98cb30870572785b5f85e339"} Jun 09 01:19:24.271979 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:24.271588 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f83e1756-56ca-42fb-a339-de1f84b4bec6","Type":"ContainerStarted","Data":"dcb446fb194d00f493ddc4610435e01344d6b33edccd425a314ceb21f5d82163"} Jun 09 01:19:24.271979 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:24.271613 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f83e1756-56ca-42fb-a339-de1f84b4bec6","Type":"ContainerStarted","Data":"fa363684ea32b91ae0c580e4641057df7f1f3770e985ef8b1e03ea3fbc41799c"} Jun 09 01:19:24.299111 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:24.299013 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.65174104 podStartE2EDuration="6.298997938s" totalImagesPullingTime="3.630836901s" totalInitContainerRuntime="16.419997ms" isStatefulPod=false podCreationTimestamp="2026-06-09 01:19:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:19:24.297113051 +0000 UTC m=+225.315417244" watchObservedRunningTime="2026-06-09 01:19:24.298997938 +0000 UTC m=+225.317302130" Jun 09 01:19:26.958589 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:26.958537 2555 patch_prober.go:28] interesting pod/image-registry-5f9bcfd4d8-m4rzd container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 01:19:26.959016 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:26.958612 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" podUID="e52b9d7c-548a-4cf5-933c-7e2e0969e797" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 01:19:29.192477 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:29.192437 2555 patch_prober.go:28] interesting pod/image-registry-5f9bcfd4d8-m4rzd container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 01:19:29.192864 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:29.192502 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" podUID="e52b9d7c-548a-4cf5-933c-7e2e0969e797" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 01:19:32.409727 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:32.409691 2555 scope.go:117] "RemoveContainer" containerID="927264b3301bb501635b803b239609fddc69026bc4743ad7f4b9a11a2ad57536" Jun 09 01:19:33.305954 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:33.305930 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:19:33.306098 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:33.306003 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" event={"ID":"0b11d16a-ae26-42b4-a44a-ac3b9d007579","Type":"ContainerStarted","Data":"b97f19c925457c4d1f6b0a2bf70f4fa4a6f271a7840993f2942af5ebc3c3969f"} Jun 09 01:19:33.306302 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:33.306282 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:19:33.323757 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:33.323706 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" podStartSLOduration=55.240896533 podStartE2EDuration="58.323691525s" totalImagesPullingTime="3.082794992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:18:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:19:33.322069945 +0000 UTC m=+234.340374155" watchObservedRunningTime="2026-06-09 01:19:33.323691525 +0000 UTC m=+234.341995717" Jun 09 01:19:33.342393 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:33.342367 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-668fdbf85c-4rqfp" Jun 09 01:19:33.520079 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:33.520046 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6657875f97-c952m"] Jun 09 01:19:33.523365 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:33.523349 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-c952m" Jun 09 01:19:33.526079 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:33.526058 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-bbv9v\"" Jun 09 01:19:33.526223 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:33.526114 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 09 01:19:33.526274 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:33.526254 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 09 01:19:33.530533 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:33.530515 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-c952m"] Jun 09 01:19:33.655314 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:33.655224 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46dd5\" (UniqueName: \"kubernetes.io/projected/1f48718e-9823-419c-9681-5ca4a58a35d1-kube-api-access-46dd5\") pod \"downloads-6657875f97-c952m\" (UID: \"1f48718e-9823-419c-9681-5ca4a58a35d1\") " pod="openshift-console/downloads-6657875f97-c952m" Jun 09 01:19:33.756180 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:33.756135 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46dd5\" (UniqueName: \"kubernetes.io/projected/1f48718e-9823-419c-9681-5ca4a58a35d1-kube-api-access-46dd5\") pod \"downloads-6657875f97-c952m\" (UID: \"1f48718e-9823-419c-9681-5ca4a58a35d1\") " pod="openshift-console/downloads-6657875f97-c952m" Jun 09 01:19:33.764752 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:33.764726 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46dd5\" (UniqueName: \"kubernetes.io/projected/1f48718e-9823-419c-9681-5ca4a58a35d1-kube-api-access-46dd5\") pod \"downloads-6657875f97-c952m\" (UID: \"1f48718e-9823-419c-9681-5ca4a58a35d1\") " pod="openshift-console/downloads-6657875f97-c952m" Jun 09 01:19:33.832986 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:33.832945 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-c952m" Jun 09 01:19:33.957791 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:33.957764 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-c952m"] Jun 09 01:19:33.960476 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:19:33.960449 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f48718e_9823_419c_9681_5ca4a58a35d1.slice/crio-4f7d2421b50385c78b141bddbaee15bff69166e2abedf26842e64bf7ce776265 WatchSource:0}: Error finding container 4f7d2421b50385c78b141bddbaee15bff69166e2abedf26842e64bf7ce776265: Status 404 returned error can't find the container with id 4f7d2421b50385c78b141bddbaee15bff69166e2abedf26842e64bf7ce776265 Jun 09 01:19:34.312311 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:34.312260 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-c952m" event={"ID":"1f48718e-9823-419c-9681-5ca4a58a35d1","Type":"ContainerStarted","Data":"4f7d2421b50385c78b141bddbaee15bff69166e2abedf26842e64bf7ce776265"} Jun 09 01:19:36.959633 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:36.959599 2555 patch_prober.go:28] interesting pod/image-registry-5f9bcfd4d8-m4rzd container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 01:19:36.960066 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:36.959657 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" podUID="e52b9d7c-548a-4cf5-933c-7e2e0969e797" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 01:19:39.043989 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.043622 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d6d76d69-mtw7t"] Jun 09 01:19:39.047199 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.047178 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.051853 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.051832 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 09 01:19:39.051963 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.051832 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 09 01:19:39.051963 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.051832 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f6xxk\"" Jun 09 01:19:39.051963 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.051879 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 09 01:19:39.051963 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.051868 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 09 01:19:39.051963 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.051833 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 09 01:19:39.055748 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.055728 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d6d76d69-mtw7t"] Jun 09 01:19:39.192612 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.192574 2555 patch_prober.go:28] interesting pod/image-registry-5f9bcfd4d8-m4rzd container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 01:19:39.192774 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.192637 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" podUID="e52b9d7c-548a-4cf5-933c-7e2e0969e797" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 01:19:39.207923 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.207894 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-config\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.208083 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.207933 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-oauth-serving-cert\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.208083 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.207968 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqxnp\" (UniqueName: \"kubernetes.io/projected/d9eb5d1a-685e-4287-8b70-e664bdef0abe-kube-api-access-kqxnp\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.208083 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.208057 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-oauth-config\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.208216 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.208098 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-service-ca\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.208216 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.208128 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-serving-cert\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.308898 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.308817 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-serving-cert\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.308898 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.308856 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-config\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.308898 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.308880 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-oauth-serving-cert\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.309163 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.308916 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqxnp\" (UniqueName: \"kubernetes.io/projected/d9eb5d1a-685e-4287-8b70-e664bdef0abe-kube-api-access-kqxnp\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.309163 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.309001 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-oauth-config\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.309163 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.309042 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-service-ca\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.309704 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.309637 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-config\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.309824 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.309788 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-oauth-serving-cert\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.309900 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.309862 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-service-ca\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.311404 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.311375 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-serving-cert\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.311689 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.311673 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-oauth-config\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.317530 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.317506 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqxnp\" (UniqueName: \"kubernetes.io/projected/d9eb5d1a-685e-4287-8b70-e664bdef0abe-kube-api-access-kqxnp\") pod \"console-7d6d76d69-mtw7t\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.361773 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.361703 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-f6xxk\"" Jun 09 01:19:39.369047 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.369007 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:39.517765 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:39.517740 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d6d76d69-mtw7t"] Jun 09 01:19:39.519766 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:19:39.519743 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9eb5d1a_685e_4287_8b70_e664bdef0abe.slice/crio-ef01e939be5b9cca019f877665384deab610e44246667703ca422a821c70af17 WatchSource:0}: Error finding container ef01e939be5b9cca019f877665384deab610e44246667703ca422a821c70af17: Status 404 returned error can't find the container with id ef01e939be5b9cca019f877665384deab610e44246667703ca422a821c70af17 Jun 09 01:19:40.342479 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:40.342435 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d6d76d69-mtw7t" event={"ID":"d9eb5d1a-685e-4287-8b70-e664bdef0abe","Type":"ContainerStarted","Data":"ef01e939be5b9cca019f877665384deab610e44246667703ca422a821c70af17"} Jun 09 01:19:46.959084 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:46.959042 2555 patch_prober.go:28] interesting pod/image-registry-5f9bcfd4d8-m4rzd container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 01:19:46.959491 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:46.959108 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" podUID="e52b9d7c-548a-4cf5-933c-7e2e0969e797" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 01:19:46.959491 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:46.959159 2555 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:19:46.959804 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:46.959747 2555 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"f1c3e91aa29df2df2b3ef3896bfb0d14be7f75412e1c10bec00e227adf88eb9b"} pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" containerMessage="Container registry failed liveness probe, will be restarted" Jun 09 01:19:46.964152 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:46.964121 2555 patch_prober.go:28] interesting pod/image-registry-5f9bcfd4d8-m4rzd container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 01:19:46.964275 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:46.964179 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" podUID="e52b9d7c-548a-4cf5-933c-7e2e0969e797" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 01:19:47.019414 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.019380 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74b98df9d4-knccq"] Jun 09 01:19:47.023311 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.023288 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.033757 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.033731 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 09 01:19:47.034841 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.034817 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74b98df9d4-knccq"] Jun 09 01:19:47.187960 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.187923 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26x4t\" (UniqueName: \"kubernetes.io/projected/5ccc1cdd-1349-458f-b157-033a472d5c46-kube-api-access-26x4t\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.188142 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.188003 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-oauth-serving-cert\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.188142 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.188039 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-trusted-ca-bundle\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.188142 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.188126 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ccc1cdd-1349-458f-b157-033a472d5c46-console-oauth-config\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.188299 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.188170 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-service-ca\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.188299 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.188251 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-console-config\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.188299 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.188286 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ccc1cdd-1349-458f-b157-033a472d5c46-console-serving-cert\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.289063 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.289020 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-console-config\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.289243 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.289079 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ccc1cdd-1349-458f-b157-033a472d5c46-console-serving-cert\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.289243 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.289130 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26x4t\" (UniqueName: \"kubernetes.io/projected/5ccc1cdd-1349-458f-b157-033a472d5c46-kube-api-access-26x4t\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.289356 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.289278 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-oauth-serving-cert\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.289405 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.289352 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-trusted-ca-bundle\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.289448 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.289426 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ccc1cdd-1349-458f-b157-033a472d5c46-console-oauth-config\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.289488 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.289452 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-service-ca\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.289935 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.289880 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-console-config\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.289935 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.289932 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-oauth-serving-cert\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.290133 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.290114 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-service-ca\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.290455 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.290425 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-trusted-ca-bundle\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.292071 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.292047 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ccc1cdd-1349-458f-b157-033a472d5c46-console-serving-cert\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.292185 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.292125 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ccc1cdd-1349-458f-b157-033a472d5c46-console-oauth-config\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.297944 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.297923 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26x4t\" (UniqueName: \"kubernetes.io/projected/5ccc1cdd-1349-458f-b157-033a472d5c46-kube-api-access-26x4t\") pod \"console-74b98df9d4-knccq\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:47.335022 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:47.334984 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:49.385154 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:49.385115 2555 generic.go:356] "Generic (PLEG): container finished" podID="0855fbf3-db84-4b7c-b61a-c5dc9a63e530" containerID="3864ac2d1329cdfd732a9567f7b71de180e53d3abccbbc15e6f61f299ebc6000" exitCode=0 Jun 09 01:19:49.385628 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:49.385195 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx" event={"ID":"0855fbf3-db84-4b7c-b61a-c5dc9a63e530","Type":"ContainerDied","Data":"3864ac2d1329cdfd732a9567f7b71de180e53d3abccbbc15e6f61f299ebc6000"} Jun 09 01:19:49.385628 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:49.385594 2555 scope.go:117] "RemoveContainer" containerID="3864ac2d1329cdfd732a9567f7b71de180e53d3abccbbc15e6f61f299ebc6000" Jun 09 01:19:49.386931 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:49.386911 2555 generic.go:356] "Generic (PLEG): container finished" podID="b9464d89-5eba-4239-aff5-5f2430ecd991" containerID="be22a7b626166ddfcf89e6ebe1afd6147aff024c078ec100e9c15fe3988f6b32" exitCode=0 Jun 09 01:19:49.387018 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:49.386962 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-jn4jq" event={"ID":"b9464d89-5eba-4239-aff5-5f2430ecd991","Type":"ContainerDied","Data":"be22a7b626166ddfcf89e6ebe1afd6147aff024c078ec100e9c15fe3988f6b32"} Jun 09 01:19:49.387288 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:49.387267 2555 scope.go:117] "RemoveContainer" containerID="be22a7b626166ddfcf89e6ebe1afd6147aff024c078ec100e9c15fe3988f6b32" Jun 09 01:19:50.314344 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:50.313917 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs\") pod \"network-metrics-daemon-8prh2\" (UID: \"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19\") " pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:19:50.316881 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:50.316811 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19-metrics-certs\") pod \"network-metrics-daemon-8prh2\" (UID: \"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19\") " pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:19:50.383161 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:50.383080 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74b98df9d4-knccq"] Jun 09 01:19:50.388638 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:19:50.388599 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ccc1cdd_1349_458f_b157_033a472d5c46.slice/crio-1942f948f74976849b4d9cd9bd8ca5c3d22d7325d74b3f476b817d111c548bdb WatchSource:0}: Error finding container 1942f948f74976849b4d9cd9bd8ca5c3d22d7325d74b3f476b817d111c548bdb: Status 404 returned error can't find the container with id 1942f948f74976849b4d9cd9bd8ca5c3d22d7325d74b3f476b817d111c548bdb Jun 09 01:19:50.395086 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:50.395054 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55bb7c4d49-f9xcx" event={"ID":"0855fbf3-db84-4b7c-b61a-c5dc9a63e530","Type":"ContainerStarted","Data":"69cdfebc2d42cd115aafc28ba83679fb78ef60a970431133a45c96682dd666aa"} Jun 09 01:19:50.397050 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:50.396964 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d6d76d69-mtw7t" event={"ID":"d9eb5d1a-685e-4287-8b70-e664bdef0abe","Type":"ContainerStarted","Data":"fbf463e42864ba029866ab65efd2814f68774fda1394a23e9ea71416e0e0e17f"} Jun 09 01:19:50.400275 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:50.400247 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-jn4jq" event={"ID":"b9464d89-5eba-4239-aff5-5f2430ecd991","Type":"ContainerStarted","Data":"8890e97a6dd392d789192bc033136ed7ea274d999da4a249435589f7b0b15ee5"} Jun 09 01:19:50.402898 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:50.402874 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-c952m" event={"ID":"1f48718e-9823-419c-9681-5ca4a58a35d1","Type":"ContainerStarted","Data":"80ddc0d5f3a9606c1eb970f2bf3d678d7ac85425d053cc9cfaf74b264754918d"} Jun 09 01:19:50.403444 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:50.403419 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6657875f97-c952m" Jun 09 01:19:50.406272 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:50.405350 2555 patch_prober.go:28] interesting pod/downloads-6657875f97-c952m container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.22:8080/\": dial tcp 10.133.0.22:8080: connect: connection refused" start-of-body= Jun 09 01:19:50.406272 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:50.405407 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-6657875f97-c952m" podUID="1f48718e-9823-419c-9681-5ca4a58a35d1" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.22:8080/\": dial tcp 10.133.0.22:8080: connect: connection refused" Jun 09 01:19:50.419861 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:50.416635 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6pwvx\"" Jun 09 01:19:50.419861 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:50.419358 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d6d76d69-mtw7t" podStartSLOduration=0.685667063 podStartE2EDuration="11.419341145s" totalImagesPullingTime="10.733674082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:19:50.417071368 +0000 UTC m=+251.435375583" watchObservedRunningTime="2026-06-09 01:19:50.419341145 +0000 UTC m=+251.437645337" Jun 09 01:19:50.422820 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:50.422801 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8prh2" Jun 09 01:19:50.472387 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:50.471897 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6657875f97-c952m" podStartSLOduration=1.129918842 podStartE2EDuration="17.471873473s" totalImagesPullingTime="16.341954631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:19:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:19:50.452277228 +0000 UTC m=+251.470581440" watchObservedRunningTime="2026-06-09 01:19:50.471873473 +0000 UTC m=+251.490177667" Jun 09 01:19:50.609844 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:50.609802 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8prh2"] Jun 09 01:19:50.612913 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:19:50.612858 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ad5e6dd_1c8b_4c90_a4b6_df145fcbbf19.slice/crio-6783c7e95c7630f194f792666c9721d9ab81f58dd86bfea077e0f7add8353043 WatchSource:0}: Error finding container 6783c7e95c7630f194f792666c9721d9ab81f58dd86bfea077e0f7add8353043: Status 404 returned error can't find the container with id 6783c7e95c7630f194f792666c9721d9ab81f58dd86bfea077e0f7add8353043 Jun 09 01:19:51.417720 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:51.417686 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8prh2" event={"ID":"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19","Type":"ContainerStarted","Data":"6783c7e95c7630f194f792666c9721d9ab81f58dd86bfea077e0f7add8353043"} Jun 09 01:19:51.418279 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:51.418255 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74b98df9d4-knccq" event={"ID":"5ccc1cdd-1349-458f-b157-033a472d5c46","Type":"ContainerStarted","Data":"92f3126a33ac27649c6c4fa7a9a5d88a925560f8c1d99375e4a440e45d30e661"} Jun 09 01:19:51.418364 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:51.418297 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74b98df9d4-knccq" event={"ID":"5ccc1cdd-1349-458f-b157-033a472d5c46","Type":"ContainerStarted","Data":"1942f948f74976849b4d9cd9bd8ca5c3d22d7325d74b3f476b817d111c548bdb"} Jun 09 01:19:51.432980 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:51.430730 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6657875f97-c952m" Jun 09 01:19:51.438690 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:51.438033 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74b98df9d4-knccq" podStartSLOduration=4.438016739 podStartE2EDuration="4.438016739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:19:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:19:51.43259676 +0000 UTC m=+252.450900983" watchObservedRunningTime="2026-06-09 01:19:51.438016739 +0000 UTC m=+252.456320933" Jun 09 01:19:52.421393 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:52.421350 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8prh2" event={"ID":"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19","Type":"ContainerStarted","Data":"b9a9df708d949fee7d7f1df2f91785f0e5e722fc67ef303134d85e9d4e0a0224"} Jun 09 01:19:52.421877 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:52.421418 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8prh2" event={"ID":"8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19","Type":"ContainerStarted","Data":"31bca55c0750a1f3fcb269d05f7bc3efee71735d3d2f252b9e382f20be66d5be"} Jun 09 01:19:52.437316 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:52.437251 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8prh2" podStartSLOduration=252.382794328 podStartE2EDuration="4m13.437230522s" totalImagesPullingTime="1.054436194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:19:52.436342482 +0000 UTC m=+253.454646701" watchObservedRunningTime="2026-06-09 01:19:52.437230522 +0000 UTC m=+253.455534718" Jun 09 01:19:56.964560 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:56.964494 2555 patch_prober.go:28] interesting pod/image-registry-5f9bcfd4d8-m4rzd container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 01:19:56.965003 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:56.964578 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" podUID="e52b9d7c-548a-4cf5-933c-7e2e0969e797" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 01:19:57.335151 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:57.335101 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:57.335459 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:57.335179 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:19:57.336876 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:57.336841 2555 patch_prober.go:28] interesting pod/console-74b98df9d4-knccq container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.24:8443/health\": dial tcp 10.133.0.24:8443: connect: connection refused" start-of-body= Jun 09 01:19:57.336985 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:57.336914 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-74b98df9d4-knccq" podUID="5ccc1cdd-1349-458f-b157-033a472d5c46" containerName="console" probeResult="failure" output="Get \"https://10.133.0.24:8443/health\": dial tcp 10.133.0.24:8443: connect: connection refused" Jun 09 01:19:59.370051 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:59.370010 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:59.370428 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:59.370074 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:19:59.371261 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:59.371238 2555 patch_prober.go:28] interesting pod/console-7d6d76d69-mtw7t container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.23:8443/health\": dial tcp 10.133.0.23:8443: connect: connection refused" start-of-body= Jun 09 01:19:59.371351 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:19:59.371302 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7d6d76d69-mtw7t" podUID="d9eb5d1a-685e-4287-8b70-e664bdef0abe" containerName="console" probeResult="failure" output="Get \"https://10.133.0.23:8443/health\": dial tcp 10.133.0.23:8443: connect: connection refused" Jun 09 01:20:06.964081 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:06.964046 2555 patch_prober.go:28] interesting pod/image-registry-5f9bcfd4d8-m4rzd container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 01:20:06.964568 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:06.964099 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" podUID="e52b9d7c-548a-4cf5-933c-7e2e0969e797" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 01:20:07.335703 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:07.335672 2555 patch_prober.go:28] interesting pod/console-74b98df9d4-knccq container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.24:8443/health\": dial tcp 10.133.0.24:8443: connect: connection refused" start-of-body= Jun 09 01:20:07.335878 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:07.335732 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-74b98df9d4-knccq" podUID="5ccc1cdd-1349-458f-b157-033a472d5c46" containerName="console" probeResult="failure" output="Get \"https://10.133.0.24:8443/health\": dial tcp 10.133.0.24:8443: connect: connection refused" Jun 09 01:20:09.370079 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:09.370040 2555 patch_prober.go:28] interesting pod/console-7d6d76d69-mtw7t container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.23:8443/health\": dial tcp 10.133.0.23:8443: connect: connection refused" start-of-body= Jun 09 01:20:09.370464 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:09.370098 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7d6d76d69-mtw7t" podUID="d9eb5d1a-685e-4287-8b70-e664bdef0abe" containerName="console" probeResult="failure" output="Get \"https://10.133.0.23:8443/health\": dial tcp 10.133.0.23:8443: connect: connection refused" Jun 09 01:20:11.980787 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:11.980741 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" podUID="e52b9d7c-548a-4cf5-933c-7e2e0969e797" containerName="registry" containerID="cri-o://f1c3e91aa29df2df2b3ef3896bfb0d14be7f75412e1c10bec00e227adf88eb9b" gracePeriod=30 Jun 09 01:20:13.511956 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:13.511831 2555 generic.go:356] "Generic (PLEG): container finished" podID="e52b9d7c-548a-4cf5-933c-7e2e0969e797" containerID="f1c3e91aa29df2df2b3ef3896bfb0d14be7f75412e1c10bec00e227adf88eb9b" exitCode=0 Jun 09 01:20:13.512524 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:13.511928 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" event={"ID":"e52b9d7c-548a-4cf5-933c-7e2e0969e797","Type":"ContainerDied","Data":"f1c3e91aa29df2df2b3ef3896bfb0d14be7f75412e1c10bec00e227adf88eb9b"} Jun 09 01:20:13.512524 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:13.512141 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" event={"ID":"e52b9d7c-548a-4cf5-933c-7e2e0969e797","Type":"ContainerStarted","Data":"7d2c608995c58f2a4202de83c84fbe1318c4959c957bacd0ca2ce459c67c73cb"} Jun 09 01:20:13.512524 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:13.512174 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:20:17.335835 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:17.335794 2555 patch_prober.go:28] interesting pod/console-74b98df9d4-knccq container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.24:8443/health\": dial tcp 10.133.0.24:8443: connect: connection refused" start-of-body= Jun 09 01:20:17.336190 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:17.335879 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-74b98df9d4-knccq" podUID="5ccc1cdd-1349-458f-b157-033a472d5c46" containerName="console" probeResult="failure" output="Get \"https://10.133.0.24:8443/health\": dial tcp 10.133.0.24:8443: connect: connection refused" Jun 09 01:20:19.369923 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:19.369884 2555 patch_prober.go:28] interesting pod/console-7d6d76d69-mtw7t container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.23:8443/health\": dial tcp 10.133.0.23:8443: connect: connection refused" start-of-body= Jun 09 01:20:19.370291 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:19.369938 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7d6d76d69-mtw7t" podUID="d9eb5d1a-685e-4287-8b70-e664bdef0abe" containerName="console" probeResult="failure" output="Get \"https://10.133.0.23:8443/health\": dial tcp 10.133.0.23:8443: connect: connection refused" Jun 09 01:20:21.009316 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:20:21.009263 2555 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5vfr5" podUID="c4cca482-39bd-4a22-bed7-e85a83b32b47" Jun 09 01:20:21.544750 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:21.544717 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5vfr5" Jun 09 01:20:24.849701 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:24.849664 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:20:24.852005 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:24.851978 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c4cca482-39bd-4a22-bed7-e85a83b32b47-metrics-tls\") pod \"dns-default-5vfr5\" (UID: \"c4cca482-39bd-4a22-bed7-e85a83b32b47\") " pod="openshift-dns/dns-default-5vfr5" Jun 09 01:20:25.148410 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:25.148326 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-j9cpl\"" Jun 09 01:20:25.156065 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:25.156042 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5vfr5" Jun 09 01:20:25.308823 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:25.308797 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5vfr5"] Jun 09 01:20:25.310421 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:20:25.310392 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4cca482_39bd_4a22_bed7_e85a83b32b47.slice/crio-a4baf11b2171ed266897416b1de99e7e4b30bb5ec60a2a8cc1408e4d9ac35449 WatchSource:0}: Error finding container a4baf11b2171ed266897416b1de99e7e4b30bb5ec60a2a8cc1408e4d9ac35449: Status 404 returned error can't find the container with id a4baf11b2171ed266897416b1de99e7e4b30bb5ec60a2a8cc1408e4d9ac35449 Jun 09 01:20:25.561779 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:25.561742 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5vfr5" event={"ID":"c4cca482-39bd-4a22-bed7-e85a83b32b47","Type":"ContainerStarted","Data":"a4baf11b2171ed266897416b1de99e7e4b30bb5ec60a2a8cc1408e4d9ac35449"} Jun 09 01:20:26.962360 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:26.962316 2555 patch_prober.go:28] interesting pod/image-registry-5f9bcfd4d8-m4rzd container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 01:20:26.962867 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:26.962399 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" podUID="e52b9d7c-548a-4cf5-933c-7e2e0969e797" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 01:20:27.335972 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:27.335932 2555 patch_prober.go:28] interesting pod/console-74b98df9d4-knccq container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.24:8443/health\": dial tcp 10.133.0.24:8443: connect: connection refused" start-of-body= Jun 09 01:20:27.336134 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:27.335986 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-74b98df9d4-knccq" podUID="5ccc1cdd-1349-458f-b157-033a472d5c46" containerName="console" probeResult="failure" output="Get \"https://10.133.0.24:8443/health\": dial tcp 10.133.0.24:8443: connect: connection refused" Jun 09 01:20:27.571729 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:27.571700 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5vfr5" event={"ID":"c4cca482-39bd-4a22-bed7-e85a83b32b47","Type":"ContainerStarted","Data":"2943fd8065bb84525ccf83dd28c38b3059b47d7ad7e6e5718524dd32ff4e4ab3"} Jun 09 01:20:28.577406 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:28.577368 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5vfr5" event={"ID":"c4cca482-39bd-4a22-bed7-e85a83b32b47","Type":"ContainerStarted","Data":"01ec4bf070857a124da112553a24c22be6f8fbafdc30038e8631f20437049fec"} Jun 09 01:20:28.577908 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:28.577476 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5vfr5" Jun 09 01:20:28.595750 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:28.595699 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5vfr5" podStartSLOduration=252.482323203 podStartE2EDuration="4m14.595682305s" totalImagesPullingTime="2.113359102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:16:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:20:28.593416785 +0000 UTC m=+289.611720977" watchObservedRunningTime="2026-06-09 01:20:28.595682305 +0000 UTC m=+289.613986496" Jun 09 01:20:29.370260 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:29.370218 2555 patch_prober.go:28] interesting pod/console-7d6d76d69-mtw7t container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.23:8443/health\": dial tcp 10.133.0.23:8443: connect: connection refused" start-of-body= Jun 09 01:20:29.370436 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:29.370277 2555 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7d6d76d69-mtw7t" podUID="d9eb5d1a-685e-4287-8b70-e664bdef0abe" containerName="console" probeResult="failure" output="Get \"https://10.133.0.23:8443/health\": dial tcp 10.133.0.23:8443: connect: connection refused" Jun 09 01:20:34.519603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:34.519564 2555 patch_prober.go:28] interesting pod/image-registry-5f9bcfd4d8-m4rzd container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 01:20:34.520034 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:34.519639 2555 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" podUID="e52b9d7c-548a-4cf5-933c-7e2e0969e797" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 01:20:36.959027 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:36.958992 2555 patch_prober.go:28] interesting pod/image-registry-5f9bcfd4d8-m4rzd container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 09 01:20:36.959472 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:36.959061 2555 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" podUID="e52b9d7c-548a-4cf5-933c-7e2e0969e797" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 01:20:37.339813 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.339777 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:20:37.343509 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.343482 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:20:37.405865 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.405832 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d6d76d69-mtw7t"] Jun 09 01:20:37.513303 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.513268 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f9557d454-jtwcq"] Jun 09 01:20:37.515970 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.515945 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.527152 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.527127 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f9557d454-jtwcq"] Jun 09 01:20:37.557313 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.557282 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-trusted-ca-bundle\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.557313 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.557310 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-oauth-serving-cert\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.557476 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.557358 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b554804-73a6-4a4a-b948-46dabe1ade54-console-oauth-config\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.557476 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.557426 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-service-ca\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.557476 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.557467 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-console-config\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.557603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.557502 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b554804-73a6-4a4a-b948-46dabe1ade54-console-serving-cert\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.557603 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.557578 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbxps\" (UniqueName: \"kubernetes.io/projected/8b554804-73a6-4a4a-b948-46dabe1ade54-kube-api-access-vbxps\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.658200 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.658113 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b554804-73a6-4a4a-b948-46dabe1ade54-console-serving-cert\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.658200 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.658176 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbxps\" (UniqueName: \"kubernetes.io/projected/8b554804-73a6-4a4a-b948-46dabe1ade54-kube-api-access-vbxps\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.658200 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.658202 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-trusted-ca-bundle\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.658535 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.658222 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-oauth-serving-cert\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.658535 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.658445 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b554804-73a6-4a4a-b948-46dabe1ade54-console-oauth-config\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.658680 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.658570 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-service-ca\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.658680 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.658668 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-console-config\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.659124 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.659095 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-oauth-serving-cert\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.659392 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.659366 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-trusted-ca-bundle\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.659392 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.659386 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-console-config\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.659745 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.659728 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-service-ca\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.660820 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.660794 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b554804-73a6-4a4a-b948-46dabe1ade54-console-oauth-config\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.660915 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.660887 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b554804-73a6-4a4a-b948-46dabe1ade54-console-serving-cert\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.667066 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.667043 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbxps\" (UniqueName: \"kubernetes.io/projected/8b554804-73a6-4a4a-b948-46dabe1ade54-kube-api-access-vbxps\") pod \"console-5f9557d454-jtwcq\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.828997 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.828953 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:37.955162 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:37.955115 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f9557d454-jtwcq"] Jun 09 01:20:37.957340 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:20:37.957311 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b554804_73a6_4a4a_b948_46dabe1ade54.slice/crio-7c956fd07047cc71a8cf48698b1f197f64bd78944616e25d41cbe2abfd835e23 WatchSource:0}: Error finding container 7c956fd07047cc71a8cf48698b1f197f64bd78944616e25d41cbe2abfd835e23: Status 404 returned error can't find the container with id 7c956fd07047cc71a8cf48698b1f197f64bd78944616e25d41cbe2abfd835e23 Jun 09 01:20:38.152842 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.152799 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 01:20:38.153198 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.153081 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="alertmanager" containerID="cri-o://25c271d4b275334da919fc0508358519bc8bd714209a2044290c5699139af98e" gracePeriod=120 Jun 09 01:20:38.153198 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.153143 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="kube-rbac-proxy-metric" containerID="cri-o://a2781225e75705ca684cd104a93626a191fdae7a98cb30870572785b5f85e339" gracePeriod=120 Jun 09 01:20:38.153351 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.153192 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="config-reloader" containerID="cri-o://52059020fcd736fce03d6469c4168cab3d674c728f863e3ca73fd24a0207bc52" gracePeriod=120 Jun 09 01:20:38.153351 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.153224 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="kube-rbac-proxy" containerID="cri-o://dcb446fb194d00f493ddc4610435e01344d6b33edccd425a314ceb21f5d82163" gracePeriod=120 Jun 09 01:20:38.153351 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.153220 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="prom-label-proxy" containerID="cri-o://30ca63b6b6e9dcacd1162b593006407f9aec73808776f0c393e0366bdae8d350" gracePeriod=120 Jun 09 01:20:38.153351 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.153176 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="kube-rbac-proxy-web" containerID="cri-o://fa363684ea32b91ae0c580e4641057df7f1f3770e985ef8b1e03ea3fbc41799c" gracePeriod=120 Jun 09 01:20:38.582752 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.582722 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5vfr5" Jun 09 01:20:38.619119 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.619024 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f9557d454-jtwcq" event={"ID":"8b554804-73a6-4a4a-b948-46dabe1ade54","Type":"ContainerStarted","Data":"cab1ca7a4a2a5729921e03e36f4bb9362652a6277b46b303427cb16dc9a4cfd1"} Jun 09 01:20:38.619119 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.619089 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f9557d454-jtwcq" event={"ID":"8b554804-73a6-4a4a-b948-46dabe1ade54","Type":"ContainerStarted","Data":"7c956fd07047cc71a8cf48698b1f197f64bd78944616e25d41cbe2abfd835e23"} Jun 09 01:20:38.622352 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.622323 2555 generic.go:356] "Generic (PLEG): container finished" podID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerID="30ca63b6b6e9dcacd1162b593006407f9aec73808776f0c393e0366bdae8d350" exitCode=0 Jun 09 01:20:38.622352 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.622349 2555 generic.go:356] "Generic (PLEG): container finished" podID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerID="a2781225e75705ca684cd104a93626a191fdae7a98cb30870572785b5f85e339" exitCode=0 Jun 09 01:20:38.622503 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.622359 2555 generic.go:356] "Generic (PLEG): container finished" podID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerID="dcb446fb194d00f493ddc4610435e01344d6b33edccd425a314ceb21f5d82163" exitCode=0 Jun 09 01:20:38.622503 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.622367 2555 generic.go:356] "Generic (PLEG): container finished" podID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerID="52059020fcd736fce03d6469c4168cab3d674c728f863e3ca73fd24a0207bc52" exitCode=0 Jun 09 01:20:38.622503 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.622375 2555 generic.go:356] "Generic (PLEG): container finished" podID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerID="25c271d4b275334da919fc0508358519bc8bd714209a2044290c5699139af98e" exitCode=0 Jun 09 01:20:38.622503 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.622392 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f83e1756-56ca-42fb-a339-de1f84b4bec6","Type":"ContainerDied","Data":"30ca63b6b6e9dcacd1162b593006407f9aec73808776f0c393e0366bdae8d350"} Jun 09 01:20:38.622503 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.622464 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f83e1756-56ca-42fb-a339-de1f84b4bec6","Type":"ContainerDied","Data":"a2781225e75705ca684cd104a93626a191fdae7a98cb30870572785b5f85e339"} Jun 09 01:20:38.622693 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.622505 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f83e1756-56ca-42fb-a339-de1f84b4bec6","Type":"ContainerDied","Data":"dcb446fb194d00f493ddc4610435e01344d6b33edccd425a314ceb21f5d82163"} Jun 09 01:20:38.622693 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.622561 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f83e1756-56ca-42fb-a339-de1f84b4bec6","Type":"ContainerDied","Data":"52059020fcd736fce03d6469c4168cab3d674c728f863e3ca73fd24a0207bc52"} Jun 09 01:20:38.622693 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.622602 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f83e1756-56ca-42fb-a339-de1f84b4bec6","Type":"ContainerDied","Data":"25c271d4b275334da919fc0508358519bc8bd714209a2044290c5699139af98e"} Jun 09 01:20:38.637893 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:38.637848 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f9557d454-jtwcq" podStartSLOduration=1.637831662 podStartE2EDuration="1.637831662s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:20:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:20:38.636426756 +0000 UTC m=+299.654730960" watchObservedRunningTime="2026-06-09 01:20:38.637831662 +0000 UTC m=+299.656135853" Jun 09 01:20:39.363948 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.363921 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:20:39.365924 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.365879 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:20:39.369903 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.369879 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:20:39.371386 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.371366 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:20:39.377683 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.377664 2555 kubelet.go:1648] "Image garbage collection succeeded" Jun 09 01:20:39.401957 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.401832 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.477342 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.477311 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f83e1756-56ca-42fb-a339-de1f84b4bec6-metrics-client-ca\") pod \"f83e1756-56ca-42fb-a339-de1f84b4bec6\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " Jun 09 01:20:39.477526 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.477359 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f83e1756-56ca-42fb-a339-de1f84b4bec6-alertmanager-main-db\") pod \"f83e1756-56ca-42fb-a339-de1f84b4bec6\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " Jun 09 01:20:39.477526 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.477404 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f83e1756-56ca-42fb-a339-de1f84b4bec6-config-out\") pod \"f83e1756-56ca-42fb-a339-de1f84b4bec6\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " Jun 09 01:20:39.477526 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.477434 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy-metric\") pod \"f83e1756-56ca-42fb-a339-de1f84b4bec6\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " Jun 09 01:20:39.477526 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.477484 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-cluster-tls-config\") pod \"f83e1756-56ca-42fb-a339-de1f84b4bec6\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " Jun 09 01:20:39.477749 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.477525 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy-web\") pod \"f83e1756-56ca-42fb-a339-de1f84b4bec6\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " Jun 09 01:20:39.477749 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.477575 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-web-config\") pod \"f83e1756-56ca-42fb-a339-de1f84b4bec6\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " Jun 09 01:20:39.477749 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.477606 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7sl58\" (UniqueName: \"kubernetes.io/projected/f83e1756-56ca-42fb-a339-de1f84b4bec6-kube-api-access-7sl58\") pod \"f83e1756-56ca-42fb-a339-de1f84b4bec6\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " Jun 09 01:20:39.477749 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.477621 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-config-volume\") pod \"f83e1756-56ca-42fb-a339-de1f84b4bec6\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " Jun 09 01:20:39.477749 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.477648 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f83e1756-56ca-42fb-a339-de1f84b4bec6-alertmanager-trusted-ca-bundle\") pod \"f83e1756-56ca-42fb-a339-de1f84b4bec6\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " Jun 09 01:20:39.477749 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.477674 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy\") pod \"f83e1756-56ca-42fb-a339-de1f84b4bec6\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " Jun 09 01:20:39.477749 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.477689 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f83e1756-56ca-42fb-a339-de1f84b4bec6-tls-assets\") pod \"f83e1756-56ca-42fb-a339-de1f84b4bec6\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " Jun 09 01:20:39.477749 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.477710 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-main-tls\") pod \"f83e1756-56ca-42fb-a339-de1f84b4bec6\" (UID: \"f83e1756-56ca-42fb-a339-de1f84b4bec6\") " Jun 09 01:20:39.477749 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.477704 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f83e1756-56ca-42fb-a339-de1f84b4bec6-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "f83e1756-56ca-42fb-a339-de1f84b4bec6" (UID: "f83e1756-56ca-42fb-a339-de1f84b4bec6"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:20:39.478184 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.477747 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f83e1756-56ca-42fb-a339-de1f84b4bec6-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "f83e1756-56ca-42fb-a339-de1f84b4bec6" (UID: "f83e1756-56ca-42fb-a339-de1f84b4bec6"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 01:20:39.478184 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.478024 2555 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f83e1756-56ca-42fb-a339-de1f84b4bec6-metrics-client-ca\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:20:39.478184 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.478046 2555 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f83e1756-56ca-42fb-a339-de1f84b4bec6-alertmanager-main-db\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:20:39.478333 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.478189 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f83e1756-56ca-42fb-a339-de1f84b4bec6-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "f83e1756-56ca-42fb-a339-de1f84b4bec6" (UID: "f83e1756-56ca-42fb-a339-de1f84b4bec6"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 01:20:39.480494 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.480412 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "f83e1756-56ca-42fb-a339-de1f84b4bec6" (UID: "f83e1756-56ca-42fb-a339-de1f84b4bec6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 01:20:39.480494 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.480431 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f83e1756-56ca-42fb-a339-de1f84b4bec6-config-out" (OuterVolumeSpecName: "config-out") pod "f83e1756-56ca-42fb-a339-de1f84b4bec6" (UID: "f83e1756-56ca-42fb-a339-de1f84b4bec6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:20:39.480802 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.480775 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "f83e1756-56ca-42fb-a339-de1f84b4bec6" (UID: "f83e1756-56ca-42fb-a339-de1f84b4bec6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 01:20:39.481232 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.481130 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "f83e1756-56ca-42fb-a339-de1f84b4bec6" (UID: "f83e1756-56ca-42fb-a339-de1f84b4bec6"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 01:20:39.481907 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.481864 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "f83e1756-56ca-42fb-a339-de1f84b4bec6" (UID: "f83e1756-56ca-42fb-a339-de1f84b4bec6"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 01:20:39.482264 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.482241 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f83e1756-56ca-42fb-a339-de1f84b4bec6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f83e1756-56ca-42fb-a339-de1f84b4bec6" (UID: "f83e1756-56ca-42fb-a339-de1f84b4bec6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:20:39.482347 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.482273 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f83e1756-56ca-42fb-a339-de1f84b4bec6-kube-api-access-7sl58" (OuterVolumeSpecName: "kube-api-access-7sl58") pod "f83e1756-56ca-42fb-a339-de1f84b4bec6" (UID: "f83e1756-56ca-42fb-a339-de1f84b4bec6"). InnerVolumeSpecName "kube-api-access-7sl58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:20:39.482754 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.482734 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-config-volume" (OuterVolumeSpecName: "config-volume") pod "f83e1756-56ca-42fb-a339-de1f84b4bec6" (UID: "f83e1756-56ca-42fb-a339-de1f84b4bec6"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 01:20:39.486831 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.486806 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "f83e1756-56ca-42fb-a339-de1f84b4bec6" (UID: "f83e1756-56ca-42fb-a339-de1f84b4bec6"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 01:20:39.492716 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.492692 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-web-config" (OuterVolumeSpecName: "web-config") pod "f83e1756-56ca-42fb-a339-de1f84b4bec6" (UID: "f83e1756-56ca-42fb-a339-de1f84b4bec6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 01:20:39.578681 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.578645 2555 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f83e1756-56ca-42fb-a339-de1f84b4bec6-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:20:39.578681 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.578677 2555 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:20:39.578883 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.578689 2555 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f83e1756-56ca-42fb-a339-de1f84b4bec6-tls-assets\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:20:39.578883 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.578699 2555 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-main-tls\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:20:39.578883 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.578708 2555 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f83e1756-56ca-42fb-a339-de1f84b4bec6-config-out\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:20:39.578883 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.578716 2555 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:20:39.578883 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.578727 2555 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-cluster-tls-config\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:20:39.578883 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.578736 2555 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:20:39.578883 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.578744 2555 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-web-config\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:20:39.578883 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.578753 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sl58\" (UniqueName: \"kubernetes.io/projected/f83e1756-56ca-42fb-a339-de1f84b4bec6-kube-api-access-7sl58\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:20:39.578883 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.578760 2555 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f83e1756-56ca-42fb-a339-de1f84b4bec6-config-volume\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:20:39.629673 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.629638 2555 generic.go:356] "Generic (PLEG): container finished" podID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerID="fa363684ea32b91ae0c580e4641057df7f1f3770e985ef8b1e03ea3fbc41799c" exitCode=0 Jun 09 01:20:39.629849 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.629734 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.629849 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.629730 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f83e1756-56ca-42fb-a339-de1f84b4bec6","Type":"ContainerDied","Data":"fa363684ea32b91ae0c580e4641057df7f1f3770e985ef8b1e03ea3fbc41799c"} Jun 09 01:20:39.629916 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.629868 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f83e1756-56ca-42fb-a339-de1f84b4bec6","Type":"ContainerDied","Data":"a8634dfa7380749915063896dec15ccc8fb2966cf84810d72ffa5acf5c0cd67d"} Jun 09 01:20:39.629916 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.629903 2555 scope.go:117] "RemoveContainer" containerID="30ca63b6b6e9dcacd1162b593006407f9aec73808776f0c393e0366bdae8d350" Jun 09 01:20:39.638179 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.638164 2555 scope.go:117] "RemoveContainer" containerID="a2781225e75705ca684cd104a93626a191fdae7a98cb30870572785b5f85e339" Jun 09 01:20:39.646921 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.646893 2555 scope.go:117] "RemoveContainer" containerID="dcb446fb194d00f493ddc4610435e01344d6b33edccd425a314ceb21f5d82163" Jun 09 01:20:39.653128 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.653100 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 01:20:39.655370 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.655351 2555 scope.go:117] "RemoveContainer" containerID="fa363684ea32b91ae0c580e4641057df7f1f3770e985ef8b1e03ea3fbc41799c" Jun 09 01:20:39.656530 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.656510 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 01:20:39.662870 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.662852 2555 scope.go:117] "RemoveContainer" containerID="52059020fcd736fce03d6469c4168cab3d674c728f863e3ca73fd24a0207bc52" Jun 09 01:20:39.674005 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.673984 2555 scope.go:117] "RemoveContainer" containerID="25c271d4b275334da919fc0508358519bc8bd714209a2044290c5699139af98e" Jun 09 01:20:39.681861 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.681834 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 01:20:39.682315 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682297 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="config-reloader" Jun 09 01:20:39.682379 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682319 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="config-reloader" Jun 09 01:20:39.682379 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682338 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="kube-rbac-proxy" Jun 09 01:20:39.682379 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682346 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="kube-rbac-proxy" Jun 09 01:20:39.682379 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682368 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="prom-label-proxy" Jun 09 01:20:39.682379 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682375 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="prom-label-proxy" Jun 09 01:20:39.682526 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682386 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="init-config-reloader" Jun 09 01:20:39.682526 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682396 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="init-config-reloader" Jun 09 01:20:39.682526 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682409 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="kube-rbac-proxy-metric" Jun 09 01:20:39.682526 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682417 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="kube-rbac-proxy-metric" Jun 09 01:20:39.682526 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682435 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="alertmanager" Jun 09 01:20:39.682526 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682442 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="alertmanager" Jun 09 01:20:39.682526 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682455 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="kube-rbac-proxy-web" Jun 09 01:20:39.682526 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682463 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="kube-rbac-proxy-web" Jun 09 01:20:39.682800 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682538 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="kube-rbac-proxy-web" Jun 09 01:20:39.682800 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682569 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="kube-rbac-proxy" Jun 09 01:20:39.682800 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682579 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="kube-rbac-proxy-metric" Jun 09 01:20:39.682800 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682587 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="config-reloader" Jun 09 01:20:39.682800 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682596 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="prom-label-proxy" Jun 09 01:20:39.682800 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682604 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" containerName="alertmanager" Jun 09 01:20:39.682800 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.682758 2555 scope.go:117] "RemoveContainer" containerID="eee142e3a00cc6ac81bb73bd24a52cce112443e7d1ae37a09b1ce18858ade79c" Jun 09 01:20:39.686329 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.686307 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.688926 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.688909 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 09 01:20:39.689211 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.689194 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 09 01:20:39.689211 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.689201 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 09 01:20:39.689362 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.689252 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 09 01:20:39.689463 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.689439 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-cl768\"" Jun 09 01:20:39.689586 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.689482 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 09 01:20:39.689586 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.689577 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 09 01:20:39.689732 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.689712 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 09 01:20:39.689795 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.689731 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 09 01:20:39.691607 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.691588 2555 scope.go:117] "RemoveContainer" containerID="30ca63b6b6e9dcacd1162b593006407f9aec73808776f0c393e0366bdae8d350" Jun 09 01:20:39.691940 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:20:39.691892 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30ca63b6b6e9dcacd1162b593006407f9aec73808776f0c393e0366bdae8d350\": container with ID starting with 30ca63b6b6e9dcacd1162b593006407f9aec73808776f0c393e0366bdae8d350 not found: ID does not exist" containerID="30ca63b6b6e9dcacd1162b593006407f9aec73808776f0c393e0366bdae8d350" Jun 09 01:20:39.692030 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.691947 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30ca63b6b6e9dcacd1162b593006407f9aec73808776f0c393e0366bdae8d350"} err="failed to get container status \"30ca63b6b6e9dcacd1162b593006407f9aec73808776f0c393e0366bdae8d350\": rpc error: code = NotFound desc = could not find container \"30ca63b6b6e9dcacd1162b593006407f9aec73808776f0c393e0366bdae8d350\": container with ID starting with 30ca63b6b6e9dcacd1162b593006407f9aec73808776f0c393e0366bdae8d350 not found: ID does not exist" Jun 09 01:20:39.692030 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.691980 2555 scope.go:117] "RemoveContainer" containerID="a2781225e75705ca684cd104a93626a191fdae7a98cb30870572785b5f85e339" Jun 09 01:20:39.692477 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:20:39.692451 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2781225e75705ca684cd104a93626a191fdae7a98cb30870572785b5f85e339\": container with ID starting with a2781225e75705ca684cd104a93626a191fdae7a98cb30870572785b5f85e339 not found: ID does not exist" containerID="a2781225e75705ca684cd104a93626a191fdae7a98cb30870572785b5f85e339" Jun 09 01:20:39.692596 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.692485 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2781225e75705ca684cd104a93626a191fdae7a98cb30870572785b5f85e339"} err="failed to get container status \"a2781225e75705ca684cd104a93626a191fdae7a98cb30870572785b5f85e339\": rpc error: code = NotFound desc = could not find container \"a2781225e75705ca684cd104a93626a191fdae7a98cb30870572785b5f85e339\": container with ID starting with a2781225e75705ca684cd104a93626a191fdae7a98cb30870572785b5f85e339 not found: ID does not exist" Jun 09 01:20:39.692596 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.692508 2555 scope.go:117] "RemoveContainer" containerID="dcb446fb194d00f493ddc4610435e01344d6b33edccd425a314ceb21f5d82163" Jun 09 01:20:39.692831 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:20:39.692811 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcb446fb194d00f493ddc4610435e01344d6b33edccd425a314ceb21f5d82163\": container with ID starting with dcb446fb194d00f493ddc4610435e01344d6b33edccd425a314ceb21f5d82163 not found: ID does not exist" containerID="dcb446fb194d00f493ddc4610435e01344d6b33edccd425a314ceb21f5d82163" Jun 09 01:20:39.692893 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.692840 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcb446fb194d00f493ddc4610435e01344d6b33edccd425a314ceb21f5d82163"} err="failed to get container status \"dcb446fb194d00f493ddc4610435e01344d6b33edccd425a314ceb21f5d82163\": rpc error: code = NotFound desc = could not find container \"dcb446fb194d00f493ddc4610435e01344d6b33edccd425a314ceb21f5d82163\": container with ID starting with dcb446fb194d00f493ddc4610435e01344d6b33edccd425a314ceb21f5d82163 not found: ID does not exist" Jun 09 01:20:39.692893 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.692863 2555 scope.go:117] "RemoveContainer" containerID="fa363684ea32b91ae0c580e4641057df7f1f3770e985ef8b1e03ea3fbc41799c" Jun 09 01:20:39.693150 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:20:39.693129 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa363684ea32b91ae0c580e4641057df7f1f3770e985ef8b1e03ea3fbc41799c\": container with ID starting with fa363684ea32b91ae0c580e4641057df7f1f3770e985ef8b1e03ea3fbc41799c not found: ID does not exist" containerID="fa363684ea32b91ae0c580e4641057df7f1f3770e985ef8b1e03ea3fbc41799c" Jun 09 01:20:39.693202 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.693176 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa363684ea32b91ae0c580e4641057df7f1f3770e985ef8b1e03ea3fbc41799c"} err="failed to get container status \"fa363684ea32b91ae0c580e4641057df7f1f3770e985ef8b1e03ea3fbc41799c\": rpc error: code = NotFound desc = could not find container \"fa363684ea32b91ae0c580e4641057df7f1f3770e985ef8b1e03ea3fbc41799c\": container with ID starting with fa363684ea32b91ae0c580e4641057df7f1f3770e985ef8b1e03ea3fbc41799c not found: ID does not exist" Jun 09 01:20:39.693254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.693201 2555 scope.go:117] "RemoveContainer" containerID="52059020fcd736fce03d6469c4168cab3d674c728f863e3ca73fd24a0207bc52" Jun 09 01:20:39.693633 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:20:39.693609 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52059020fcd736fce03d6469c4168cab3d674c728f863e3ca73fd24a0207bc52\": container with ID starting with 52059020fcd736fce03d6469c4168cab3d674c728f863e3ca73fd24a0207bc52 not found: ID does not exist" containerID="52059020fcd736fce03d6469c4168cab3d674c728f863e3ca73fd24a0207bc52" Jun 09 01:20:39.693728 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.693639 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52059020fcd736fce03d6469c4168cab3d674c728f863e3ca73fd24a0207bc52"} err="failed to get container status \"52059020fcd736fce03d6469c4168cab3d674c728f863e3ca73fd24a0207bc52\": rpc error: code = NotFound desc = could not find container \"52059020fcd736fce03d6469c4168cab3d674c728f863e3ca73fd24a0207bc52\": container with ID starting with 52059020fcd736fce03d6469c4168cab3d674c728f863e3ca73fd24a0207bc52 not found: ID does not exist" Jun 09 01:20:39.693728 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.693660 2555 scope.go:117] "RemoveContainer" containerID="25c271d4b275334da919fc0508358519bc8bd714209a2044290c5699139af98e" Jun 09 01:20:39.693938 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:20:39.693915 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25c271d4b275334da919fc0508358519bc8bd714209a2044290c5699139af98e\": container with ID starting with 25c271d4b275334da919fc0508358519bc8bd714209a2044290c5699139af98e not found: ID does not exist" containerID="25c271d4b275334da919fc0508358519bc8bd714209a2044290c5699139af98e" Jun 09 01:20:39.694015 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.693938 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25c271d4b275334da919fc0508358519bc8bd714209a2044290c5699139af98e"} err="failed to get container status \"25c271d4b275334da919fc0508358519bc8bd714209a2044290c5699139af98e\": rpc error: code = NotFound desc = could not find container \"25c271d4b275334da919fc0508358519bc8bd714209a2044290c5699139af98e\": container with ID starting with 25c271d4b275334da919fc0508358519bc8bd714209a2044290c5699139af98e not found: ID does not exist" Jun 09 01:20:39.694015 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.693955 2555 scope.go:117] "RemoveContainer" containerID="eee142e3a00cc6ac81bb73bd24a52cce112443e7d1ae37a09b1ce18858ade79c" Jun 09 01:20:39.694176 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.694159 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 09 01:20:39.694228 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:20:39.694196 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eee142e3a00cc6ac81bb73bd24a52cce112443e7d1ae37a09b1ce18858ade79c\": container with ID starting with eee142e3a00cc6ac81bb73bd24a52cce112443e7d1ae37a09b1ce18858ade79c not found: ID does not exist" containerID="eee142e3a00cc6ac81bb73bd24a52cce112443e7d1ae37a09b1ce18858ade79c" Jun 09 01:20:39.694272 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.694223 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eee142e3a00cc6ac81bb73bd24a52cce112443e7d1ae37a09b1ce18858ade79c"} err="failed to get container status \"eee142e3a00cc6ac81bb73bd24a52cce112443e7d1ae37a09b1ce18858ade79c\": rpc error: code = NotFound desc = could not find container \"eee142e3a00cc6ac81bb73bd24a52cce112443e7d1ae37a09b1ce18858ade79c\": container with ID starting with eee142e3a00cc6ac81bb73bd24a52cce112443e7d1ae37a09b1ce18858ade79c not found: ID does not exist" Jun 09 01:20:39.698283 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.698259 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 01:20:39.780450 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.780406 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.780450 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.780453 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78pd9\" (UniqueName: \"kubernetes.io/projected/ccaa03a9-10a6-4e89-bed6-206d2dd63032-kube-api-access-78pd9\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.780695 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.780484 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.780695 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.780561 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ccaa03a9-10a6-4e89-bed6-206d2dd63032-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.780695 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.780603 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ccaa03a9-10a6-4e89-bed6-206d2dd63032-config-out\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.780695 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.780622 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.780695 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.780644 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.780695 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.780668 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccaa03a9-10a6-4e89-bed6-206d2dd63032-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.780909 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.780700 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccaa03a9-10a6-4e89-bed6-206d2dd63032-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.780909 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.780740 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-config-volume\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.780909 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.780758 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ccaa03a9-10a6-4e89-bed6-206d2dd63032-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.780909 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.780779 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.780909 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.780800 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-web-config\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.882068 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.881970 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.882068 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.882020 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78pd9\" (UniqueName: \"kubernetes.io/projected/ccaa03a9-10a6-4e89-bed6-206d2dd63032-kube-api-access-78pd9\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.882257 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.882148 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.882257 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.882208 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ccaa03a9-10a6-4e89-bed6-206d2dd63032-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.882348 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.882256 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ccaa03a9-10a6-4e89-bed6-206d2dd63032-config-out\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.882348 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.882281 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.882348 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.882309 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.882348 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.882332 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccaa03a9-10a6-4e89-bed6-206d2dd63032-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.882573 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.882367 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccaa03a9-10a6-4e89-bed6-206d2dd63032-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.882573 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.882410 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-config-volume\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.882573 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.882434 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ccaa03a9-10a6-4e89-bed6-206d2dd63032-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.882573 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.882464 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.882573 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.882490 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-web-config\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.885095 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.885068 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ccaa03a9-10a6-4e89-bed6-206d2dd63032-config-out\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.885218 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.885179 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.885288 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.885274 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.885349 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.885286 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.885349 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.885340 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ccaa03a9-10a6-4e89-bed6-206d2dd63032-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.885449 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.885365 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.885532 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.885506 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ccaa03a9-10a6-4e89-bed6-206d2dd63032-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.885824 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.885801 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-web-config\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.885971 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.885951 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-config-volume\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.886048 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.885979 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ccaa03a9-10a6-4e89-bed6-206d2dd63032-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.886146 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.886126 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ccaa03a9-10a6-4e89-bed6-206d2dd63032-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.887321 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.887295 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ccaa03a9-10a6-4e89-bed6-206d2dd63032-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.889794 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.889777 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78pd9\" (UniqueName: \"kubernetes.io/projected/ccaa03a9-10a6-4e89-bed6-206d2dd63032-kube-api-access-78pd9\") pod \"alertmanager-main-0\" (UID: \"ccaa03a9-10a6-4e89-bed6-206d2dd63032\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:39.995984 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:39.995930 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 09 01:20:40.129886 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:40.129746 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 01:20:40.131744 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:20:40.131717 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccaa03a9_10a6_4e89_bed6_206d2dd63032.slice/crio-e6dae58572ad89a25199034f1d3676d1b9d0f787ee78afbe321d55a0ca7ec28e WatchSource:0}: Error finding container e6dae58572ad89a25199034f1d3676d1b9d0f787ee78afbe321d55a0ca7ec28e: Status 404 returned error can't find the container with id e6dae58572ad89a25199034f1d3676d1b9d0f787ee78afbe321d55a0ca7ec28e Jun 09 01:20:40.635246 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:40.635210 2555 generic.go:356] "Generic (PLEG): container finished" podID="ccaa03a9-10a6-4e89-bed6-206d2dd63032" containerID="26a24a9f2a97508f228cae0d03b8813d9ac2455ec17959bc6c712cdf01f8ca8a" exitCode=0 Jun 09 01:20:40.635690 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:40.635293 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccaa03a9-10a6-4e89-bed6-206d2dd63032","Type":"ContainerDied","Data":"26a24a9f2a97508f228cae0d03b8813d9ac2455ec17959bc6c712cdf01f8ca8a"} Jun 09 01:20:40.635690 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:40.635376 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccaa03a9-10a6-4e89-bed6-206d2dd63032","Type":"ContainerStarted","Data":"e6dae58572ad89a25199034f1d3676d1b9d0f787ee78afbe321d55a0ca7ec28e"} Jun 09 01:20:41.415380 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:41.415343 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f83e1756-56ca-42fb-a339-de1f84b4bec6" path="/var/lib/kubelet/pods/f83e1756-56ca-42fb-a339-de1f84b4bec6/volumes" Jun 09 01:20:41.644718 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:41.644672 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccaa03a9-10a6-4e89-bed6-206d2dd63032","Type":"ContainerStarted","Data":"9a1e6ebdeb673a3fb238bfd76f932feed3014f05fd9b425cff330738099879ff"} Jun 09 01:20:41.645082 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:41.644766 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccaa03a9-10a6-4e89-bed6-206d2dd63032","Type":"ContainerStarted","Data":"4572eeb4d553cf8b59a2153ac1e7a5009f138e4a045e005129208e54657fc8fd"} Jun 09 01:20:41.645082 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:41.644803 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccaa03a9-10a6-4e89-bed6-206d2dd63032","Type":"ContainerStarted","Data":"91586d40742a7932331a2ece0298aed6763ad78fc049e81c90f5ec5f1138612a"} Jun 09 01:20:41.645082 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:41.644838 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccaa03a9-10a6-4e89-bed6-206d2dd63032","Type":"ContainerStarted","Data":"4791e61f45d013ceb6b26af0123221ea456c6cf802deba0777f7973035637d63"} Jun 09 01:20:41.645082 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:41.644880 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccaa03a9-10a6-4e89-bed6-206d2dd63032","Type":"ContainerStarted","Data":"7fe3de5959ce465d61f07a383f5c37f124dc38ce713538d085449770244c2e46"} Jun 09 01:20:41.645082 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:41.644920 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ccaa03a9-10a6-4e89-bed6-206d2dd63032","Type":"ContainerStarted","Data":"cb5508a2176385366c9cf7326572c73c146bafe7bd6cfb8207ac645b8ff3bc3f"} Jun 09 01:20:41.673103 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:41.672995 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.657966937 podStartE2EDuration="2.672972984s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.006047ms" isStatefulPod=false podCreationTimestamp="2026-06-09 01:20:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:20:41.669996156 +0000 UTC m=+302.688300389" watchObservedRunningTime="2026-06-09 01:20:41.672972984 +0000 UTC m=+302.691277177" Jun 09 01:20:44.520026 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:44.519996 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5f9bcfd4d8-m4rzd" Jun 09 01:20:47.829503 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:47.829472 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:47.829908 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:47.829529 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:47.834472 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:47.834446 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:48.687110 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:48.687079 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:20:48.735970 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:20:48.735940 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74b98df9d4-knccq"] Jun 09 01:21:02.425829 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.425788 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7d6d76d69-mtw7t" podUID="d9eb5d1a-685e-4287-8b70-e664bdef0abe" containerName="console" containerID="cri-o://fbf463e42864ba029866ab65efd2814f68774fda1394a23e9ea71416e0e0e17f" gracePeriod=15 Jun 09 01:21:02.673282 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.673253 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:21:02.746794 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.746704 2555 generic.go:356] "Generic (PLEG): container finished" podID="d9eb5d1a-685e-4287-8b70-e664bdef0abe" containerID="fbf463e42864ba029866ab65efd2814f68774fda1394a23e9ea71416e0e0e17f" exitCode=0 Jun 09 01:21:02.746794 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.746751 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d6d76d69-mtw7t" Jun 09 01:21:02.746963 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.746788 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d6d76d69-mtw7t" event={"ID":"d9eb5d1a-685e-4287-8b70-e664bdef0abe","Type":"ContainerDied","Data":"fbf463e42864ba029866ab65efd2814f68774fda1394a23e9ea71416e0e0e17f"} Jun 09 01:21:02.746963 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.746841 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d6d76d69-mtw7t" event={"ID":"d9eb5d1a-685e-4287-8b70-e664bdef0abe","Type":"ContainerDied","Data":"ef01e939be5b9cca019f877665384deab610e44246667703ca422a821c70af17"} Jun 09 01:21:02.746963 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.746867 2555 scope.go:117] "RemoveContainer" containerID="fbf463e42864ba029866ab65efd2814f68774fda1394a23e9ea71416e0e0e17f" Jun 09 01:21:02.754823 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.754807 2555 scope.go:117] "RemoveContainer" containerID="fbf463e42864ba029866ab65efd2814f68774fda1394a23e9ea71416e0e0e17f" Jun 09 01:21:02.755052 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:21:02.755035 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbf463e42864ba029866ab65efd2814f68774fda1394a23e9ea71416e0e0e17f\": container with ID starting with fbf463e42864ba029866ab65efd2814f68774fda1394a23e9ea71416e0e0e17f not found: ID does not exist" containerID="fbf463e42864ba029866ab65efd2814f68774fda1394a23e9ea71416e0e0e17f" Jun 09 01:21:02.755108 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.755061 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbf463e42864ba029866ab65efd2814f68774fda1394a23e9ea71416e0e0e17f"} err="failed to get container status \"fbf463e42864ba029866ab65efd2814f68774fda1394a23e9ea71416e0e0e17f\": rpc error: code = NotFound desc = could not find container \"fbf463e42864ba029866ab65efd2814f68774fda1394a23e9ea71416e0e0e17f\": container with ID starting with fbf463e42864ba029866ab65efd2814f68774fda1394a23e9ea71416e0e0e17f not found: ID does not exist" Jun 09 01:21:02.794110 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.794074 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-config\") pod \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " Jun 09 01:21:02.794265 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.794139 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-serving-cert\") pod \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " Jun 09 01:21:02.794265 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.794177 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-service-ca\") pod \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " Jun 09 01:21:02.794265 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.794214 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-oauth-config\") pod \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " Jun 09 01:21:02.794265 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.794262 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-oauth-serving-cert\") pod \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " Jun 09 01:21:02.794460 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.794294 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqxnp\" (UniqueName: \"kubernetes.io/projected/d9eb5d1a-685e-4287-8b70-e664bdef0abe-kube-api-access-kqxnp\") pod \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\" (UID: \"d9eb5d1a-685e-4287-8b70-e664bdef0abe\") " Jun 09 01:21:02.794676 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.794643 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-config" (OuterVolumeSpecName: "console-config") pod "d9eb5d1a-685e-4287-8b70-e664bdef0abe" (UID: "d9eb5d1a-685e-4287-8b70-e664bdef0abe"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 01:21:02.794774 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.794659 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-service-ca" (OuterVolumeSpecName: "service-ca") pod "d9eb5d1a-685e-4287-8b70-e664bdef0abe" (UID: "d9eb5d1a-685e-4287-8b70-e664bdef0abe"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 01:21:02.794774 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.794756 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d9eb5d1a-685e-4287-8b70-e664bdef0abe" (UID: "d9eb5d1a-685e-4287-8b70-e664bdef0abe"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 01:21:02.796500 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.796466 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d9eb5d1a-685e-4287-8b70-e664bdef0abe" (UID: "d9eb5d1a-685e-4287-8b70-e664bdef0abe"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 01:21:02.796500 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.796473 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9eb5d1a-685e-4287-8b70-e664bdef0abe-kube-api-access-kqxnp" (OuterVolumeSpecName: "kube-api-access-kqxnp") pod "d9eb5d1a-685e-4287-8b70-e664bdef0abe" (UID: "d9eb5d1a-685e-4287-8b70-e664bdef0abe"). InnerVolumeSpecName "kube-api-access-kqxnp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:21:02.796693 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.796513 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d9eb5d1a-685e-4287-8b70-e664bdef0abe" (UID: "d9eb5d1a-685e-4287-8b70-e664bdef0abe"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 01:21:02.895585 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.895534 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-oauth-config\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:21:02.895585 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.895579 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-oauth-serving-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:21:02.895585 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.895589 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqxnp\" (UniqueName: \"kubernetes.io/projected/d9eb5d1a-685e-4287-8b70-e664bdef0abe-kube-api-access-kqxnp\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:21:02.895801 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.895598 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-config\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:21:02.895801 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.895607 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d9eb5d1a-685e-4287-8b70-e664bdef0abe-console-serving-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:21:02.895801 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:02.895617 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d9eb5d1a-685e-4287-8b70-e664bdef0abe-service-ca\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:21:03.068854 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:03.068827 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d6d76d69-mtw7t"] Jun 09 01:21:03.072601 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:03.072578 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d6d76d69-mtw7t"] Jun 09 01:21:03.414891 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:03.414805 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9eb5d1a-685e-4287-8b70-e664bdef0abe" path="/var/lib/kubelet/pods/d9eb5d1a-685e-4287-8b70-e664bdef0abe/volumes" Jun 09 01:21:13.757034 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:13.756983 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-74b98df9d4-knccq" podUID="5ccc1cdd-1349-458f-b157-033a472d5c46" containerName="console" containerID="cri-o://92f3126a33ac27649c6c4fa7a9a5d88a925560f8c1d99375e4a440e45d30e661" gracePeriod=15 Jun 09 01:21:14.004520 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.004498 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:21:14.094980 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.094947 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ccc1cdd-1349-458f-b157-033a472d5c46-console-oauth-config\") pod \"5ccc1cdd-1349-458f-b157-033a472d5c46\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " Jun 09 01:21:14.094980 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.094987 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-oauth-serving-cert\") pod \"5ccc1cdd-1349-458f-b157-033a472d5c46\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " Jun 09 01:21:14.095235 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.095006 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26x4t\" (UniqueName: \"kubernetes.io/projected/5ccc1cdd-1349-458f-b157-033a472d5c46-kube-api-access-26x4t\") pod \"5ccc1cdd-1349-458f-b157-033a472d5c46\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " Jun 09 01:21:14.095235 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.095117 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ccc1cdd-1349-458f-b157-033a472d5c46-console-serving-cert\") pod \"5ccc1cdd-1349-458f-b157-033a472d5c46\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " Jun 09 01:21:14.095235 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.095181 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-trusted-ca-bundle\") pod \"5ccc1cdd-1349-458f-b157-033a472d5c46\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " Jun 09 01:21:14.095235 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.095218 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-service-ca\") pod \"5ccc1cdd-1349-458f-b157-033a472d5c46\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " Jun 09 01:21:14.095441 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.095243 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-console-config\") pod \"5ccc1cdd-1349-458f-b157-033a472d5c46\" (UID: \"5ccc1cdd-1349-458f-b157-033a472d5c46\") " Jun 09 01:21:14.095493 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.095399 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5ccc1cdd-1349-458f-b157-033a472d5c46" (UID: "5ccc1cdd-1349-458f-b157-033a472d5c46"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 01:21:14.095597 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.095566 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-service-ca" (OuterVolumeSpecName: "service-ca") pod "5ccc1cdd-1349-458f-b157-033a472d5c46" (UID: "5ccc1cdd-1349-458f-b157-033a472d5c46"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 01:21:14.095670 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.095646 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-oauth-serving-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:21:14.095909 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.095816 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-console-config" (OuterVolumeSpecName: "console-config") pod "5ccc1cdd-1349-458f-b157-033a472d5c46" (UID: "5ccc1cdd-1349-458f-b157-033a472d5c46"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 01:21:14.095981 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.095964 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5ccc1cdd-1349-458f-b157-033a472d5c46" (UID: "5ccc1cdd-1349-458f-b157-033a472d5c46"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 01:21:14.097095 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.097069 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ccc1cdd-1349-458f-b157-033a472d5c46-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5ccc1cdd-1349-458f-b157-033a472d5c46" (UID: "5ccc1cdd-1349-458f-b157-033a472d5c46"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 01:21:14.097452 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.097427 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ccc1cdd-1349-458f-b157-033a472d5c46-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5ccc1cdd-1349-458f-b157-033a472d5c46" (UID: "5ccc1cdd-1349-458f-b157-033a472d5c46"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 01:21:14.097524 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.097444 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ccc1cdd-1349-458f-b157-033a472d5c46-kube-api-access-26x4t" (OuterVolumeSpecName: "kube-api-access-26x4t") pod "5ccc1cdd-1349-458f-b157-033a472d5c46" (UID: "5ccc1cdd-1349-458f-b157-033a472d5c46"). InnerVolumeSpecName "kube-api-access-26x4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:21:14.197061 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.197026 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ccc1cdd-1349-458f-b157-033a472d5c46-console-oauth-config\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:21:14.197061 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.197055 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26x4t\" (UniqueName: \"kubernetes.io/projected/5ccc1cdd-1349-458f-b157-033a472d5c46-kube-api-access-26x4t\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:21:14.197061 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.197065 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ccc1cdd-1349-458f-b157-033a472d5c46-console-serving-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:21:14.197285 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.197075 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-trusted-ca-bundle\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:21:14.197285 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.197084 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-service-ca\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:21:14.197285 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.197092 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ccc1cdd-1349-458f-b157-033a472d5c46-console-config\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:21:14.804682 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.804644 2555 generic.go:356] "Generic (PLEG): container finished" podID="5ccc1cdd-1349-458f-b157-033a472d5c46" containerID="92f3126a33ac27649c6c4fa7a9a5d88a925560f8c1d99375e4a440e45d30e661" exitCode=0 Jun 09 01:21:14.805085 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.804694 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74b98df9d4-knccq" Jun 09 01:21:14.805085 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.804715 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74b98df9d4-knccq" event={"ID":"5ccc1cdd-1349-458f-b157-033a472d5c46","Type":"ContainerDied","Data":"92f3126a33ac27649c6c4fa7a9a5d88a925560f8c1d99375e4a440e45d30e661"} Jun 09 01:21:14.805085 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.804763 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74b98df9d4-knccq" event={"ID":"5ccc1cdd-1349-458f-b157-033a472d5c46","Type":"ContainerDied","Data":"1942f948f74976849b4d9cd9bd8ca5c3d22d7325d74b3f476b817d111c548bdb"} Jun 09 01:21:14.805085 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.804791 2555 scope.go:117] "RemoveContainer" containerID="92f3126a33ac27649c6c4fa7a9a5d88a925560f8c1d99375e4a440e45d30e661" Jun 09 01:21:14.813278 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.813262 2555 scope.go:117] "RemoveContainer" containerID="92f3126a33ac27649c6c4fa7a9a5d88a925560f8c1d99375e4a440e45d30e661" Jun 09 01:21:14.813505 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:21:14.813489 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92f3126a33ac27649c6c4fa7a9a5d88a925560f8c1d99375e4a440e45d30e661\": container with ID starting with 92f3126a33ac27649c6c4fa7a9a5d88a925560f8c1d99375e4a440e45d30e661 not found: ID does not exist" containerID="92f3126a33ac27649c6c4fa7a9a5d88a925560f8c1d99375e4a440e45d30e661" Jun 09 01:21:14.813571 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.813513 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92f3126a33ac27649c6c4fa7a9a5d88a925560f8c1d99375e4a440e45d30e661"} err="failed to get container status \"92f3126a33ac27649c6c4fa7a9a5d88a925560f8c1d99375e4a440e45d30e661\": rpc error: code = NotFound desc = could not find container \"92f3126a33ac27649c6c4fa7a9a5d88a925560f8c1d99375e4a440e45d30e661\": container with ID starting with 92f3126a33ac27649c6c4fa7a9a5d88a925560f8c1d99375e4a440e45d30e661 not found: ID does not exist" Jun 09 01:21:14.825163 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.825139 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74b98df9d4-knccq"] Jun 09 01:21:14.829450 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:14.829431 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74b98df9d4-knccq"] Jun 09 01:21:15.415948 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:15.415914 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ccc1cdd-1349-458f-b157-033a472d5c46" path="/var/lib/kubelet/pods/5ccc1cdd-1349-458f-b157-033a472d5c46/volumes" Jun 09 01:21:32.742449 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.742413 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2"] Jun 09 01:21:32.742853 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.742734 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ccc1cdd-1349-458f-b157-033a472d5c46" containerName="console" Jun 09 01:21:32.742853 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.742746 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ccc1cdd-1349-458f-b157-033a472d5c46" containerName="console" Jun 09 01:21:32.742853 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.742774 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9eb5d1a-685e-4287-8b70-e664bdef0abe" containerName="console" Jun 09 01:21:32.742853 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.742780 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9eb5d1a-685e-4287-8b70-e664bdef0abe" containerName="console" Jun 09 01:21:32.742853 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.742840 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9eb5d1a-685e-4287-8b70-e664bdef0abe" containerName="console" Jun 09 01:21:32.742853 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.742851 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ccc1cdd-1349-458f-b157-033a472d5c46" containerName="console" Jun 09 01:21:32.746002 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.745985 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" Jun 09 01:21:32.749210 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.749188 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6rf6w\"" Jun 09 01:21:32.749344 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.749315 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 09 01:21:32.749420 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.749403 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 09 01:21:32.754945 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.754923 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2"] Jun 09 01:21:32.864599 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.864559 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwfpb\" (UniqueName: \"kubernetes.io/projected/a65a6c7b-3810-4ee3-9c32-147db153e787-kube-api-access-wwfpb\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2\" (UID: \"a65a6c7b-3810-4ee3-9c32-147db153e787\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" Jun 09 01:21:32.864814 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.864692 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a65a6c7b-3810-4ee3-9c32-147db153e787-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2\" (UID: \"a65a6c7b-3810-4ee3-9c32-147db153e787\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" Jun 09 01:21:32.864814 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.864745 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a65a6c7b-3810-4ee3-9c32-147db153e787-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2\" (UID: \"a65a6c7b-3810-4ee3-9c32-147db153e787\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" Jun 09 01:21:32.965636 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.965602 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwfpb\" (UniqueName: \"kubernetes.io/projected/a65a6c7b-3810-4ee3-9c32-147db153e787-kube-api-access-wwfpb\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2\" (UID: \"a65a6c7b-3810-4ee3-9c32-147db153e787\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" Jun 09 01:21:32.965805 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.965682 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a65a6c7b-3810-4ee3-9c32-147db153e787-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2\" (UID: \"a65a6c7b-3810-4ee3-9c32-147db153e787\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" Jun 09 01:21:32.965805 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.965712 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a65a6c7b-3810-4ee3-9c32-147db153e787-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2\" (UID: \"a65a6c7b-3810-4ee3-9c32-147db153e787\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" Jun 09 01:21:32.966086 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.966066 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a65a6c7b-3810-4ee3-9c32-147db153e787-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2\" (UID: \"a65a6c7b-3810-4ee3-9c32-147db153e787\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" Jun 09 01:21:32.966121 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.966096 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a65a6c7b-3810-4ee3-9c32-147db153e787-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2\" (UID: \"a65a6c7b-3810-4ee3-9c32-147db153e787\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" Jun 09 01:21:32.974187 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:32.974157 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwfpb\" (UniqueName: \"kubernetes.io/projected/a65a6c7b-3810-4ee3-9c32-147db153e787-kube-api-access-wwfpb\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2\" (UID: \"a65a6c7b-3810-4ee3-9c32-147db153e787\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" Jun 09 01:21:33.056629 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:33.056601 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" Jun 09 01:21:33.181129 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:33.181105 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2"] Jun 09 01:21:33.183260 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:21:33.183231 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda65a6c7b_3810_4ee3_9c32_147db153e787.slice/crio-61a2a34320b57fa285f129157cd6eeb1d727c0f609c98d204b88b2a3f51c4c85 WatchSource:0}: Error finding container 61a2a34320b57fa285f129157cd6eeb1d727c0f609c98d204b88b2a3f51c4c85: Status 404 returned error can't find the container with id 61a2a34320b57fa285f129157cd6eeb1d727c0f609c98d204b88b2a3f51c4c85 Jun 09 01:21:33.185167 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:33.185151 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 01:21:33.890663 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:33.890619 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" event={"ID":"a65a6c7b-3810-4ee3-9c32-147db153e787","Type":"ContainerStarted","Data":"61a2a34320b57fa285f129157cd6eeb1d727c0f609c98d204b88b2a3f51c4c85"} Jun 09 01:21:44.945586 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:44.945518 2555 generic.go:356] "Generic (PLEG): container finished" podID="a65a6c7b-3810-4ee3-9c32-147db153e787" containerID="d575ccdfe4f590124ac18fef2a86ff1bbdccd8f5078fc51f37f088252fbfa465" exitCode=0 Jun 09 01:21:44.946046 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:44.945603 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" event={"ID":"a65a6c7b-3810-4ee3-9c32-147db153e787","Type":"ContainerDied","Data":"d575ccdfe4f590124ac18fef2a86ff1bbdccd8f5078fc51f37f088252fbfa465"} Jun 09 01:21:47.960090 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:47.960046 2555 generic.go:356] "Generic (PLEG): container finished" podID="a65a6c7b-3810-4ee3-9c32-147db153e787" containerID="fca687783b79fab688d5dffd94c737f63cc905e09f6ce07482954b5635615223" exitCode=0 Jun 09 01:21:47.960459 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:47.960100 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" event={"ID":"a65a6c7b-3810-4ee3-9c32-147db153e787","Type":"ContainerDied","Data":"fca687783b79fab688d5dffd94c737f63cc905e09f6ce07482954b5635615223"} Jun 09 01:21:58.007291 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:58.007252 2555 generic.go:356] "Generic (PLEG): container finished" podID="a65a6c7b-3810-4ee3-9c32-147db153e787" containerID="170eed6f1d8df9a2ee9c3df521b09e4873d5f4adae18d91d962643cba6a74b9c" exitCode=0 Jun 09 01:21:58.007291 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:58.007291 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" event={"ID":"a65a6c7b-3810-4ee3-9c32-147db153e787","Type":"ContainerDied","Data":"170eed6f1d8df9a2ee9c3df521b09e4873d5f4adae18d91d962643cba6a74b9c"} Jun 09 01:21:59.143752 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:59.143723 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" Jun 09 01:21:59.201670 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:59.201632 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a65a6c7b-3810-4ee3-9c32-147db153e787-bundle\") pod \"a65a6c7b-3810-4ee3-9c32-147db153e787\" (UID: \"a65a6c7b-3810-4ee3-9c32-147db153e787\") " Jun 09 01:21:59.201860 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:59.201714 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a65a6c7b-3810-4ee3-9c32-147db153e787-util\") pod \"a65a6c7b-3810-4ee3-9c32-147db153e787\" (UID: \"a65a6c7b-3810-4ee3-9c32-147db153e787\") " Jun 09 01:21:59.201860 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:59.201751 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwfpb\" (UniqueName: \"kubernetes.io/projected/a65a6c7b-3810-4ee3-9c32-147db153e787-kube-api-access-wwfpb\") pod \"a65a6c7b-3810-4ee3-9c32-147db153e787\" (UID: \"a65a6c7b-3810-4ee3-9c32-147db153e787\") " Jun 09 01:21:59.202288 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:59.202262 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a65a6c7b-3810-4ee3-9c32-147db153e787-bundle" (OuterVolumeSpecName: "bundle") pod "a65a6c7b-3810-4ee3-9c32-147db153e787" (UID: "a65a6c7b-3810-4ee3-9c32-147db153e787"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:21:59.203926 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:59.203889 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a65a6c7b-3810-4ee3-9c32-147db153e787-kube-api-access-wwfpb" (OuterVolumeSpecName: "kube-api-access-wwfpb") pod "a65a6c7b-3810-4ee3-9c32-147db153e787" (UID: "a65a6c7b-3810-4ee3-9c32-147db153e787"). InnerVolumeSpecName "kube-api-access-wwfpb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:21:59.205654 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:59.205632 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a65a6c7b-3810-4ee3-9c32-147db153e787-util" (OuterVolumeSpecName: "util") pod "a65a6c7b-3810-4ee3-9c32-147db153e787" (UID: "a65a6c7b-3810-4ee3-9c32-147db153e787"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:21:59.302890 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:59.302858 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a65a6c7b-3810-4ee3-9c32-147db153e787-util\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:21:59.302890 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:59.302889 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwfpb\" (UniqueName: \"kubernetes.io/projected/a65a6c7b-3810-4ee3-9c32-147db153e787-kube-api-access-wwfpb\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:21:59.303078 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:21:59.302903 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a65a6c7b-3810-4ee3-9c32-147db153e787-bundle\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:22:00.016394 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:00.016369 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" Jun 09 01:22:00.016394 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:00.016389 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19dbs6f2" event={"ID":"a65a6c7b-3810-4ee3-9c32-147db153e787","Type":"ContainerDied","Data":"61a2a34320b57fa285f129157cd6eeb1d727c0f609c98d204b88b2a3f51c4c85"} Jun 09 01:22:00.016621 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:00.016432 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61a2a34320b57fa285f129157cd6eeb1d727c0f609c98d204b88b2a3f51c4c85" Jun 09 01:22:15.020508 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.020471 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-v9cz8"] Jun 09 01:22:15.020987 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.020820 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a65a6c7b-3810-4ee3-9c32-147db153e787" containerName="pull" Jun 09 01:22:15.020987 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.020833 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a65a6c7b-3810-4ee3-9c32-147db153e787" containerName="pull" Jun 09 01:22:15.020987 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.020851 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a65a6c7b-3810-4ee3-9c32-147db153e787" containerName="util" Jun 09 01:22:15.020987 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.020856 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a65a6c7b-3810-4ee3-9c32-147db153e787" containerName="util" Jun 09 01:22:15.020987 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.020871 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a65a6c7b-3810-4ee3-9c32-147db153e787" containerName="extract" Jun 09 01:22:15.020987 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.020878 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="a65a6c7b-3810-4ee3-9c32-147db153e787" containerName="extract" Jun 09 01:22:15.020987 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.020930 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="a65a6c7b-3810-4ee3-9c32-147db153e787" containerName="extract" Jun 09 01:22:15.024131 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.024107 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-v9cz8" Jun 09 01:22:15.027084 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.027058 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 09 01:22:15.027659 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.027630 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 09 01:22:15.027885 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.027639 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-tzbzp\"" Jun 09 01:22:15.037592 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.037567 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-v9cz8"] Jun 09 01:22:15.141920 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.141884 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2nlq\" (UniqueName: \"kubernetes.io/projected/e144ecf6-0714-4282-8fd8-cb970e8f1a74-kube-api-access-d2nlq\") pod \"cert-manager-operator-controller-manager-57757f7cfc-v9cz8\" (UID: \"e144ecf6-0714-4282-8fd8-cb970e8f1a74\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-v9cz8" Jun 09 01:22:15.142089 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.141929 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e144ecf6-0714-4282-8fd8-cb970e8f1a74-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-v9cz8\" (UID: \"e144ecf6-0714-4282-8fd8-cb970e8f1a74\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-v9cz8" Jun 09 01:22:15.242987 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.242950 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2nlq\" (UniqueName: \"kubernetes.io/projected/e144ecf6-0714-4282-8fd8-cb970e8f1a74-kube-api-access-d2nlq\") pod \"cert-manager-operator-controller-manager-57757f7cfc-v9cz8\" (UID: \"e144ecf6-0714-4282-8fd8-cb970e8f1a74\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-v9cz8" Jun 09 01:22:15.243172 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.243002 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e144ecf6-0714-4282-8fd8-cb970e8f1a74-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-v9cz8\" (UID: \"e144ecf6-0714-4282-8fd8-cb970e8f1a74\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-v9cz8" Jun 09 01:22:15.243368 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.243352 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e144ecf6-0714-4282-8fd8-cb970e8f1a74-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-v9cz8\" (UID: \"e144ecf6-0714-4282-8fd8-cb970e8f1a74\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-v9cz8" Jun 09 01:22:15.251590 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.251562 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2nlq\" (UniqueName: \"kubernetes.io/projected/e144ecf6-0714-4282-8fd8-cb970e8f1a74-kube-api-access-d2nlq\") pod \"cert-manager-operator-controller-manager-57757f7cfc-v9cz8\" (UID: \"e144ecf6-0714-4282-8fd8-cb970e8f1a74\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-v9cz8" Jun 09 01:22:15.339834 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.339798 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-v9cz8" Jun 09 01:22:15.470559 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:15.470521 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-v9cz8"] Jun 09 01:22:15.472312 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:22:15.472285 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode144ecf6_0714_4282_8fd8_cb970e8f1a74.slice/crio-2a95754d5b961927d8b80948baaa38408d0da96bd59a05a8cc4ab186083dc881 WatchSource:0}: Error finding container 2a95754d5b961927d8b80948baaa38408d0da96bd59a05a8cc4ab186083dc881: Status 404 returned error can't find the container with id 2a95754d5b961927d8b80948baaa38408d0da96bd59a05a8cc4ab186083dc881 Jun 09 01:22:16.087883 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:16.087848 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-v9cz8" event={"ID":"e144ecf6-0714-4282-8fd8-cb970e8f1a74","Type":"ContainerStarted","Data":"2a95754d5b961927d8b80948baaa38408d0da96bd59a05a8cc4ab186083dc881"} Jun 09 01:22:18.099458 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:18.099425 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-v9cz8" event={"ID":"e144ecf6-0714-4282-8fd8-cb970e8f1a74","Type":"ContainerStarted","Data":"cb25e5ede9a03e91e26355dc677c8938d529e5f74a8f9c915160131d08d689b4"} Jun 09 01:22:18.124789 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:18.124721 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-v9cz8" podStartSLOduration=2.405449542 podStartE2EDuration="4.124702985s" totalImagesPullingTime="1.719253443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:22:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:22:18.122920083 +0000 UTC m=+399.141224292" watchObservedRunningTime="2026-06-09 01:22:18.124702985 +0000 UTC m=+399.143007176" Jun 09 01:22:19.807962 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.807925 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-jxfn9"] Jun 09 01:22:19.817478 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.817445 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-jxfn9" Jun 09 01:22:19.819926 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.819903 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-jxfn9"] Jun 09 01:22:19.820466 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.820451 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 09 01:22:19.820622 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.820603 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 09 01:22:19.871840 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.871806 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv"] Jun 09 01:22:19.874378 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.874359 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" Jun 09 01:22:19.877461 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.877436 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 09 01:22:19.877594 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.877436 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 09 01:22:19.877594 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.877449 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6rf6w\"" Jun 09 01:22:19.883484 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.883459 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv"] Jun 09 01:22:19.886964 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.886934 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e973b15-d72a-42fc-86af-3c331b731ca9-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-jxfn9\" (UID: \"1e973b15-d72a-42fc-86af-3c331b731ca9\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-jxfn9" Jun 09 01:22:19.887094 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.886986 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvnhc\" (UniqueName: \"kubernetes.io/projected/1e973b15-d72a-42fc-86af-3c331b731ca9-kube-api-access-dvnhc\") pod \"cert-manager-webhook-7dc76cbb94-jxfn9\" (UID: \"1e973b15-d72a-42fc-86af-3c331b731ca9\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-jxfn9" Jun 09 01:22:19.987663 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.987629 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6df80b48-2740-4049-9934-671861099711-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv\" (UID: \"6df80b48-2740-4049-9934-671861099711\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" Jun 09 01:22:19.987839 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.987673 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e973b15-d72a-42fc-86af-3c331b731ca9-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-jxfn9\" (UID: \"1e973b15-d72a-42fc-86af-3c331b731ca9\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-jxfn9" Jun 09 01:22:19.987839 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.987700 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvnhc\" (UniqueName: \"kubernetes.io/projected/1e973b15-d72a-42fc-86af-3c331b731ca9-kube-api-access-dvnhc\") pod \"cert-manager-webhook-7dc76cbb94-jxfn9\" (UID: \"1e973b15-d72a-42fc-86af-3c331b731ca9\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-jxfn9" Jun 09 01:22:19.987839 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.987778 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6df80b48-2740-4049-9934-671861099711-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv\" (UID: \"6df80b48-2740-4049-9934-671861099711\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" Jun 09 01:22:19.987839 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.987820 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w78vx\" (UniqueName: \"kubernetes.io/projected/6df80b48-2740-4049-9934-671861099711-kube-api-access-w78vx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv\" (UID: \"6df80b48-2740-4049-9934-671861099711\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" Jun 09 01:22:19.995982 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.995953 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1e973b15-d72a-42fc-86af-3c331b731ca9-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-jxfn9\" (UID: \"1e973b15-d72a-42fc-86af-3c331b731ca9\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-jxfn9" Jun 09 01:22:19.996082 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:19.996064 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvnhc\" (UniqueName: \"kubernetes.io/projected/1e973b15-d72a-42fc-86af-3c331b731ca9-kube-api-access-dvnhc\") pod \"cert-manager-webhook-7dc76cbb94-jxfn9\" (UID: \"1e973b15-d72a-42fc-86af-3c331b731ca9\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-jxfn9" Jun 09 01:22:20.089114 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:20.089012 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w78vx\" (UniqueName: \"kubernetes.io/projected/6df80b48-2740-4049-9934-671861099711-kube-api-access-w78vx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv\" (UID: \"6df80b48-2740-4049-9934-671861099711\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" Jun 09 01:22:20.089114 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:20.089075 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6df80b48-2740-4049-9934-671861099711-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv\" (UID: \"6df80b48-2740-4049-9934-671861099711\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" Jun 09 01:22:20.089350 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:20.089127 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6df80b48-2740-4049-9934-671861099711-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv\" (UID: \"6df80b48-2740-4049-9934-671861099711\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" Jun 09 01:22:20.089441 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:20.089425 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6df80b48-2740-4049-9934-671861099711-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv\" (UID: \"6df80b48-2740-4049-9934-671861099711\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" Jun 09 01:22:20.089493 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:20.089473 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6df80b48-2740-4049-9934-671861099711-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv\" (UID: \"6df80b48-2740-4049-9934-671861099711\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" Jun 09 01:22:20.097900 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:20.097878 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w78vx\" (UniqueName: \"kubernetes.io/projected/6df80b48-2740-4049-9934-671861099711-kube-api-access-w78vx\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv\" (UID: \"6df80b48-2740-4049-9934-671861099711\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" Jun 09 01:22:20.138868 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:20.138833 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-jxfn9" Jun 09 01:22:20.184560 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:20.184509 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" Jun 09 01:22:20.269386 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:20.269360 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-jxfn9"] Jun 09 01:22:20.271031 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:22:20.271001 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e973b15_d72a_42fc_86af_3c331b731ca9.slice/crio-8389a6bd1dc74aef9a06f464388646818f8f065035c1ca50ca84762b527b827e WatchSource:0}: Error finding container 8389a6bd1dc74aef9a06f464388646818f8f065035c1ca50ca84762b527b827e: Status 404 returned error can't find the container with id 8389a6bd1dc74aef9a06f464388646818f8f065035c1ca50ca84762b527b827e Jun 09 01:22:20.323142 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:20.323120 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv"] Jun 09 01:22:20.325064 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:22:20.325037 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6df80b48_2740_4049_9934_671861099711.slice/crio-595bc7d49deae7270e62f05c28cb9213618fe79c436ec89be73ce5080c17ccea WatchSource:0}: Error finding container 595bc7d49deae7270e62f05c28cb9213618fe79c436ec89be73ce5080c17ccea: Status 404 returned error can't find the container with id 595bc7d49deae7270e62f05c28cb9213618fe79c436ec89be73ce5080c17ccea Jun 09 01:22:21.113598 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:21.113537 2555 generic.go:356] "Generic (PLEG): container finished" podID="6df80b48-2740-4049-9934-671861099711" containerID="853750b14314708eab5a67be6942f95bf9c09184fc32a13e7398d7622bfffd42" exitCode=0 Jun 09 01:22:21.113598 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:21.113581 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" event={"ID":"6df80b48-2740-4049-9934-671861099711","Type":"ContainerDied","Data":"853750b14314708eab5a67be6942f95bf9c09184fc32a13e7398d7622bfffd42"} Jun 09 01:22:21.114124 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:21.113645 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" event={"ID":"6df80b48-2740-4049-9934-671861099711","Type":"ContainerStarted","Data":"595bc7d49deae7270e62f05c28cb9213618fe79c436ec89be73ce5080c17ccea"} Jun 09 01:22:21.114815 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:21.114797 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-jxfn9" event={"ID":"1e973b15-d72a-42fc-86af-3c331b731ca9","Type":"ContainerStarted","Data":"8389a6bd1dc74aef9a06f464388646818f8f065035c1ca50ca84762b527b827e"} Jun 09 01:22:24.132013 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:24.131981 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-jxfn9" event={"ID":"1e973b15-d72a-42fc-86af-3c331b731ca9","Type":"ContainerStarted","Data":"785c2ac4c64fefd19e3eaaab246f09e7e6cd667b5a91a7e6e532edc254e8df66"} Jun 09 01:22:24.132367 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:24.132032 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-jxfn9" Jun 09 01:22:24.147816 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:24.147765 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-jxfn9" podStartSLOduration=1.387071346 podStartE2EDuration="5.147750257s" totalImagesPullingTime="3.760678911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:22:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:22:24.147304546 +0000 UTC m=+405.165608740" watchObservedRunningTime="2026-06-09 01:22:24.147750257 +0000 UTC m=+405.166054448" Jun 09 01:22:26.143373 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:26.143337 2555 generic.go:356] "Generic (PLEG): container finished" podID="6df80b48-2740-4049-9934-671861099711" containerID="5abc073eab85b35a5d4d5c109ec93402a621d0476a250fa18e8c14dc267a0fd0" exitCode=0 Jun 09 01:22:26.143762 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:26.143381 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" event={"ID":"6df80b48-2740-4049-9934-671861099711","Type":"ContainerDied","Data":"5abc073eab85b35a5d4d5c109ec93402a621d0476a250fa18e8c14dc267a0fd0"} Jun 09 01:22:27.150062 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:27.150023 2555 generic.go:356] "Generic (PLEG): container finished" podID="6df80b48-2740-4049-9934-671861099711" containerID="e4169587b726e94eb1ccc4c71f2857fa57158bea563d03db9d6c29c82b7f0689" exitCode=0 Jun 09 01:22:27.150418 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:27.150109 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" event={"ID":"6df80b48-2740-4049-9934-671861099711","Type":"ContainerDied","Data":"e4169587b726e94eb1ccc4c71f2857fa57158bea563d03db9d6c29c82b7f0689"} Jun 09 01:22:28.280712 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:28.280684 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" Jun 09 01:22:28.362009 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:28.361974 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6df80b48-2740-4049-9934-671861099711-bundle\") pod \"6df80b48-2740-4049-9934-671861099711\" (UID: \"6df80b48-2740-4049-9934-671861099711\") " Jun 09 01:22:28.362152 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:28.362034 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6df80b48-2740-4049-9934-671861099711-util\") pod \"6df80b48-2740-4049-9934-671861099711\" (UID: \"6df80b48-2740-4049-9934-671861099711\") " Jun 09 01:22:28.362152 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:28.362061 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w78vx\" (UniqueName: \"kubernetes.io/projected/6df80b48-2740-4049-9934-671861099711-kube-api-access-w78vx\") pod \"6df80b48-2740-4049-9934-671861099711\" (UID: \"6df80b48-2740-4049-9934-671861099711\") " Jun 09 01:22:28.362459 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:28.362431 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6df80b48-2740-4049-9934-671861099711-bundle" (OuterVolumeSpecName: "bundle") pod "6df80b48-2740-4049-9934-671861099711" (UID: "6df80b48-2740-4049-9934-671861099711"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:22:28.364169 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:28.364142 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6df80b48-2740-4049-9934-671861099711-kube-api-access-w78vx" (OuterVolumeSpecName: "kube-api-access-w78vx") pod "6df80b48-2740-4049-9934-671861099711" (UID: "6df80b48-2740-4049-9934-671861099711"). InnerVolumeSpecName "kube-api-access-w78vx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:22:28.366463 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:28.366444 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6df80b48-2740-4049-9934-671861099711-util" (OuterVolumeSpecName: "util") pod "6df80b48-2740-4049-9934-671861099711" (UID: "6df80b48-2740-4049-9934-671861099711"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:22:28.462992 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:28.462912 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6df80b48-2740-4049-9934-671861099711-bundle\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:22:28.462992 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:28.462937 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6df80b48-2740-4049-9934-671861099711-util\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:22:28.462992 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:28.462946 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w78vx\" (UniqueName: \"kubernetes.io/projected/6df80b48-2740-4049-9934-671861099711-kube-api-access-w78vx\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:22:29.160702 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:29.160652 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" event={"ID":"6df80b48-2740-4049-9934-671861099711","Type":"ContainerDied","Data":"595bc7d49deae7270e62f05c28cb9213618fe79c436ec89be73ce5080c17ccea"} Jun 09 01:22:29.160702 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:29.160681 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fws2pv" Jun 09 01:22:29.160944 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:29.160713 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="595bc7d49deae7270e62f05c28cb9213618fe79c436ec89be73ce5080c17ccea" Jun 09 01:22:30.139567 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:30.139510 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-jxfn9" Jun 09 01:22:37.069775 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.069742 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dh9t"] Jun 09 01:22:37.070206 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.070073 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6df80b48-2740-4049-9934-671861099711" containerName="pull" Jun 09 01:22:37.070206 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.070084 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6df80b48-2740-4049-9934-671861099711" containerName="pull" Jun 09 01:22:37.070206 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.070099 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6df80b48-2740-4049-9934-671861099711" containerName="extract" Jun 09 01:22:37.070206 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.070104 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6df80b48-2740-4049-9934-671861099711" containerName="extract" Jun 09 01:22:37.070206 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.070121 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6df80b48-2740-4049-9934-671861099711" containerName="util" Jun 09 01:22:37.070206 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.070127 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="6df80b48-2740-4049-9934-671861099711" containerName="util" Jun 09 01:22:37.070206 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.070176 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="6df80b48-2740-4049-9934-671861099711" containerName="extract" Jun 09 01:22:37.072437 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.072418 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dh9t" Jun 09 01:22:37.075325 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.075307 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 09 01:22:37.075439 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.075359 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 09 01:22:37.076715 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.076698 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-krzdw\"" Jun 09 01:22:37.081683 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.081663 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dh9t"] Jun 09 01:22:37.240952 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.240911 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e772b6e0-56bd-48ef-ac3e-00b39d17c61c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2dh9t\" (UID: \"e772b6e0-56bd-48ef-ac3e-00b39d17c61c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dh9t" Jun 09 01:22:37.241130 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.241034 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sjf8\" (UniqueName: \"kubernetes.io/projected/e772b6e0-56bd-48ef-ac3e-00b39d17c61c-kube-api-access-8sjf8\") pod \"openshift-lws-operator-fd8ccff4c-2dh9t\" (UID: \"e772b6e0-56bd-48ef-ac3e-00b39d17c61c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dh9t" Jun 09 01:22:37.341731 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.341643 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e772b6e0-56bd-48ef-ac3e-00b39d17c61c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2dh9t\" (UID: \"e772b6e0-56bd-48ef-ac3e-00b39d17c61c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dh9t" Jun 09 01:22:37.341874 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.341748 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sjf8\" (UniqueName: \"kubernetes.io/projected/e772b6e0-56bd-48ef-ac3e-00b39d17c61c-kube-api-access-8sjf8\") pod \"openshift-lws-operator-fd8ccff4c-2dh9t\" (UID: \"e772b6e0-56bd-48ef-ac3e-00b39d17c61c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dh9t" Jun 09 01:22:37.342056 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.342033 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e772b6e0-56bd-48ef-ac3e-00b39d17c61c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2dh9t\" (UID: \"e772b6e0-56bd-48ef-ac3e-00b39d17c61c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dh9t" Jun 09 01:22:37.350632 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.350607 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sjf8\" (UniqueName: \"kubernetes.io/projected/e772b6e0-56bd-48ef-ac3e-00b39d17c61c-kube-api-access-8sjf8\") pod \"openshift-lws-operator-fd8ccff4c-2dh9t\" (UID: \"e772b6e0-56bd-48ef-ac3e-00b39d17c61c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dh9t" Jun 09 01:22:37.382131 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.382084 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dh9t" Jun 09 01:22:37.509790 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:37.509766 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dh9t"] Jun 09 01:22:37.511555 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:22:37.511514 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode772b6e0_56bd_48ef_ac3e_00b39d17c61c.slice/crio-228f90a4f9e400576d2f042bebf040e019b6bbcc91cd6e4cdf78053e3e9cb5dd WatchSource:0}: Error finding container 228f90a4f9e400576d2f042bebf040e019b6bbcc91cd6e4cdf78053e3e9cb5dd: Status 404 returned error can't find the container with id 228f90a4f9e400576d2f042bebf040e019b6bbcc91cd6e4cdf78053e3e9cb5dd Jun 09 01:22:38.203333 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:38.203290 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dh9t" event={"ID":"e772b6e0-56bd-48ef-ac3e-00b39d17c61c","Type":"ContainerStarted","Data":"228f90a4f9e400576d2f042bebf040e019b6bbcc91cd6e4cdf78053e3e9cb5dd"} Jun 09 01:22:38.755157 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:38.755121 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-59c77684cc-qvp8p"] Jun 09 01:22:38.757961 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:38.757944 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-qvp8p" Jun 09 01:22:38.760589 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:38.760564 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-4f5cs\"" Jun 09 01:22:38.767066 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:38.767047 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-qvp8p"] Jun 09 01:22:38.854351 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:38.854312 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ade294fe-054d-478d-96a3-1fd981016034-bound-sa-token\") pod \"cert-manager-59c77684cc-qvp8p\" (UID: \"ade294fe-054d-478d-96a3-1fd981016034\") " pod="cert-manager/cert-manager-59c77684cc-qvp8p" Jun 09 01:22:38.854536 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:38.854405 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xkzg\" (UniqueName: \"kubernetes.io/projected/ade294fe-054d-478d-96a3-1fd981016034-kube-api-access-7xkzg\") pod \"cert-manager-59c77684cc-qvp8p\" (UID: \"ade294fe-054d-478d-96a3-1fd981016034\") " pod="cert-manager/cert-manager-59c77684cc-qvp8p" Jun 09 01:22:38.955150 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:38.955115 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xkzg\" (UniqueName: \"kubernetes.io/projected/ade294fe-054d-478d-96a3-1fd981016034-kube-api-access-7xkzg\") pod \"cert-manager-59c77684cc-qvp8p\" (UID: \"ade294fe-054d-478d-96a3-1fd981016034\") " pod="cert-manager/cert-manager-59c77684cc-qvp8p" Jun 09 01:22:38.955326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:38.955183 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ade294fe-054d-478d-96a3-1fd981016034-bound-sa-token\") pod \"cert-manager-59c77684cc-qvp8p\" (UID: \"ade294fe-054d-478d-96a3-1fd981016034\") " pod="cert-manager/cert-manager-59c77684cc-qvp8p" Jun 09 01:22:38.963386 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:38.963356 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ade294fe-054d-478d-96a3-1fd981016034-bound-sa-token\") pod \"cert-manager-59c77684cc-qvp8p\" (UID: \"ade294fe-054d-478d-96a3-1fd981016034\") " pod="cert-manager/cert-manager-59c77684cc-qvp8p" Jun 09 01:22:38.963526 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:38.963442 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xkzg\" (UniqueName: \"kubernetes.io/projected/ade294fe-054d-478d-96a3-1fd981016034-kube-api-access-7xkzg\") pod \"cert-manager-59c77684cc-qvp8p\" (UID: \"ade294fe-054d-478d-96a3-1fd981016034\") " pod="cert-manager/cert-manager-59c77684cc-qvp8p" Jun 09 01:22:39.068567 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:39.068495 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-qvp8p" Jun 09 01:22:39.197698 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:39.197673 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-qvp8p"] Jun 09 01:22:39.199579 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:22:39.199537 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podade294fe_054d_478d_96a3_1fd981016034.slice/crio-78b7218b6ba3b910ab692a0651398c8504a88336813f8fa9c258c66793fdabea WatchSource:0}: Error finding container 78b7218b6ba3b910ab692a0651398c8504a88336813f8fa9c258c66793fdabea: Status 404 returned error can't find the container with id 78b7218b6ba3b910ab692a0651398c8504a88336813f8fa9c258c66793fdabea Jun 09 01:22:39.210111 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:39.210085 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-qvp8p" event={"ID":"ade294fe-054d-478d-96a3-1fd981016034","Type":"ContainerStarted","Data":"78b7218b6ba3b910ab692a0651398c8504a88336813f8fa9c258c66793fdabea"} Jun 09 01:22:40.216932 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:40.216887 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-qvp8p" event={"ID":"ade294fe-054d-478d-96a3-1fd981016034","Type":"ContainerStarted","Data":"c09c51cd54241d6b98f2c3d8ec44ed4e00e02457aec6f8dfa42a3d9ea3104589"} Jun 09 01:22:40.234897 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:40.234849 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-59c77684cc-qvp8p" podStartSLOduration=2.234835542 podStartE2EDuration="2.234835542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:22:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:22:40.233526507 +0000 UTC m=+421.251830701" watchObservedRunningTime="2026-06-09 01:22:40.234835542 +0000 UTC m=+421.253139734" Jun 09 01:22:42.226793 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:42.226761 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dh9t" event={"ID":"e772b6e0-56bd-48ef-ac3e-00b39d17c61c","Type":"ContainerStarted","Data":"29286b7cd7399a35a7e75f4ce37e93cb7e1b41d0bc23c78837f77b4d183263a6"} Jun 09 01:22:42.245710 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:42.245659 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dh9t" podStartSLOduration=1.092346533 podStartE2EDuration="5.245642593s" totalImagesPullingTime="4.15329606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:22:42.244847183 +0000 UTC m=+423.263151377" watchObservedRunningTime="2026-06-09 01:22:42.245642593 +0000 UTC m=+423.263946785" Jun 09 01:22:44.686976 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:44.686940 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68"] Jun 09 01:22:44.689811 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:44.689793 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" Jun 09 01:22:44.692685 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:44.692664 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 09 01:22:44.692809 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:44.692666 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6rf6w\"" Jun 09 01:22:44.694015 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:44.693996 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 09 01:22:44.697623 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:44.697601 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68"] Jun 09 01:22:44.702354 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:44.702333 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fbf409f-fb3b-400a-866d-33f8fee5c970-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68\" (UID: \"3fbf409f-fb3b-400a-866d-33f8fee5c970\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" Jun 09 01:22:44.702443 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:44.702376 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm79b\" (UniqueName: \"kubernetes.io/projected/3fbf409f-fb3b-400a-866d-33f8fee5c970-kube-api-access-jm79b\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68\" (UID: \"3fbf409f-fb3b-400a-866d-33f8fee5c970\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" Jun 09 01:22:44.702443 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:44.702416 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fbf409f-fb3b-400a-866d-33f8fee5c970-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68\" (UID: \"3fbf409f-fb3b-400a-866d-33f8fee5c970\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" Jun 09 01:22:44.803718 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:44.803670 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fbf409f-fb3b-400a-866d-33f8fee5c970-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68\" (UID: \"3fbf409f-fb3b-400a-866d-33f8fee5c970\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" Jun 09 01:22:44.803891 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:44.803741 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm79b\" (UniqueName: \"kubernetes.io/projected/3fbf409f-fb3b-400a-866d-33f8fee5c970-kube-api-access-jm79b\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68\" (UID: \"3fbf409f-fb3b-400a-866d-33f8fee5c970\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" Jun 09 01:22:44.803891 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:44.803770 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fbf409f-fb3b-400a-866d-33f8fee5c970-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68\" (UID: \"3fbf409f-fb3b-400a-866d-33f8fee5c970\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" Jun 09 01:22:44.804069 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:44.804051 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fbf409f-fb3b-400a-866d-33f8fee5c970-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68\" (UID: \"3fbf409f-fb3b-400a-866d-33f8fee5c970\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" Jun 09 01:22:44.804132 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:44.804118 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fbf409f-fb3b-400a-866d-33f8fee5c970-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68\" (UID: \"3fbf409f-fb3b-400a-866d-33f8fee5c970\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" Jun 09 01:22:44.812578 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:44.812526 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm79b\" (UniqueName: \"kubernetes.io/projected/3fbf409f-fb3b-400a-866d-33f8fee5c970-kube-api-access-jm79b\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68\" (UID: \"3fbf409f-fb3b-400a-866d-33f8fee5c970\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" Jun 09 01:22:44.999765 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:44.999661 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" Jun 09 01:22:45.133640 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:45.133608 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68"] Jun 09 01:22:45.135505 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:22:45.135479 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fbf409f_fb3b_400a_866d_33f8fee5c970.slice/crio-f3df87c9787a4f6ed1bf0181457c9b04f1519d697747eca80e808af618cc95a3 WatchSource:0}: Error finding container f3df87c9787a4f6ed1bf0181457c9b04f1519d697747eca80e808af618cc95a3: Status 404 returned error can't find the container with id f3df87c9787a4f6ed1bf0181457c9b04f1519d697747eca80e808af618cc95a3 Jun 09 01:22:45.251302 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:45.251233 2555 generic.go:356] "Generic (PLEG): container finished" podID="3fbf409f-fb3b-400a-866d-33f8fee5c970" containerID="1096ebb82e5d26a7c6524d4898ce5c445990b9a1721c03bdcf8f7ed33d2acc41" exitCode=0 Jun 09 01:22:45.251418 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:45.251312 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" event={"ID":"3fbf409f-fb3b-400a-866d-33f8fee5c970","Type":"ContainerDied","Data":"1096ebb82e5d26a7c6524d4898ce5c445990b9a1721c03bdcf8f7ed33d2acc41"} Jun 09 01:22:45.251418 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:45.251385 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" event={"ID":"3fbf409f-fb3b-400a-866d-33f8fee5c970","Type":"ContainerStarted","Data":"f3df87c9787a4f6ed1bf0181457c9b04f1519d697747eca80e808af618cc95a3"} Jun 09 01:22:46.257533 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:46.257438 2555 generic.go:356] "Generic (PLEG): container finished" podID="3fbf409f-fb3b-400a-866d-33f8fee5c970" containerID="d3af9f3b26b89f7b59ea8f112d14a2adb09a4a55ca30c349f583cae2e1a45b52" exitCode=0 Jun 09 01:22:46.257533 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:46.257482 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" event={"ID":"3fbf409f-fb3b-400a-866d-33f8fee5c970","Type":"ContainerDied","Data":"d3af9f3b26b89f7b59ea8f112d14a2adb09a4a55ca30c349f583cae2e1a45b52"} Jun 09 01:22:47.264890 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:47.264857 2555 generic.go:356] "Generic (PLEG): container finished" podID="3fbf409f-fb3b-400a-866d-33f8fee5c970" containerID="c90f6e1d6224cae596f2ad340258deec3fb979b7d6c2e216222b59bca79fb723" exitCode=0 Jun 09 01:22:47.264890 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:47.264895 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" event={"ID":"3fbf409f-fb3b-400a-866d-33f8fee5c970","Type":"ContainerDied","Data":"c90f6e1d6224cae596f2ad340258deec3fb979b7d6c2e216222b59bca79fb723"} Jun 09 01:22:48.400688 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:48.400662 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" Jun 09 01:22:48.437133 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:48.437105 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fbf409f-fb3b-400a-866d-33f8fee5c970-util\") pod \"3fbf409f-fb3b-400a-866d-33f8fee5c970\" (UID: \"3fbf409f-fb3b-400a-866d-33f8fee5c970\") " Jun 09 01:22:48.437318 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:48.437205 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jm79b\" (UniqueName: \"kubernetes.io/projected/3fbf409f-fb3b-400a-866d-33f8fee5c970-kube-api-access-jm79b\") pod \"3fbf409f-fb3b-400a-866d-33f8fee5c970\" (UID: \"3fbf409f-fb3b-400a-866d-33f8fee5c970\") " Jun 09 01:22:48.437318 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:48.437243 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fbf409f-fb3b-400a-866d-33f8fee5c970-bundle\") pod \"3fbf409f-fb3b-400a-866d-33f8fee5c970\" (UID: \"3fbf409f-fb3b-400a-866d-33f8fee5c970\") " Jun 09 01:22:48.438001 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:48.437975 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fbf409f-fb3b-400a-866d-33f8fee5c970-bundle" (OuterVolumeSpecName: "bundle") pod "3fbf409f-fb3b-400a-866d-33f8fee5c970" (UID: "3fbf409f-fb3b-400a-866d-33f8fee5c970"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:22:48.439205 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:48.439185 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fbf409f-fb3b-400a-866d-33f8fee5c970-kube-api-access-jm79b" (OuterVolumeSpecName: "kube-api-access-jm79b") pod "3fbf409f-fb3b-400a-866d-33f8fee5c970" (UID: "3fbf409f-fb3b-400a-866d-33f8fee5c970"). InnerVolumeSpecName "kube-api-access-jm79b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:22:48.442515 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:48.442480 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fbf409f-fb3b-400a-866d-33f8fee5c970-util" (OuterVolumeSpecName: "util") pod "3fbf409f-fb3b-400a-866d-33f8fee5c970" (UID: "3fbf409f-fb3b-400a-866d-33f8fee5c970"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:22:48.538450 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:48.538411 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jm79b\" (UniqueName: \"kubernetes.io/projected/3fbf409f-fb3b-400a-866d-33f8fee5c970-kube-api-access-jm79b\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:22:48.538450 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:48.538443 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fbf409f-fb3b-400a-866d-33f8fee5c970-bundle\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:22:48.538450 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:48.538452 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fbf409f-fb3b-400a-866d-33f8fee5c970-util\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:22:49.277077 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:49.277045 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" Jun 09 01:22:49.277244 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:49.277074 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdjqs68" event={"ID":"3fbf409f-fb3b-400a-866d-33f8fee5c970","Type":"ContainerDied","Data":"f3df87c9787a4f6ed1bf0181457c9b04f1519d697747eca80e808af618cc95a3"} Jun 09 01:22:49.277244 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:49.277120 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f3df87c9787a4f6ed1bf0181457c9b04f1519d697747eca80e808af618cc95a3" Jun 09 01:22:57.419074 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.419039 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb"] Jun 09 01:22:57.419439 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.419357 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fbf409f-fb3b-400a-866d-33f8fee5c970" containerName="pull" Jun 09 01:22:57.419439 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.419367 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fbf409f-fb3b-400a-866d-33f8fee5c970" containerName="pull" Jun 09 01:22:57.419439 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.419382 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fbf409f-fb3b-400a-866d-33f8fee5c970" containerName="extract" Jun 09 01:22:57.419439 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.419388 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fbf409f-fb3b-400a-866d-33f8fee5c970" containerName="extract" Jun 09 01:22:57.419439 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.419397 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fbf409f-fb3b-400a-866d-33f8fee5c970" containerName="util" Jun 09 01:22:57.419439 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.419402 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fbf409f-fb3b-400a-866d-33f8fee5c970" containerName="util" Jun 09 01:22:57.419635 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.419467 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fbf409f-fb3b-400a-866d-33f8fee5c970" containerName="extract" Jun 09 01:22:57.423576 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.423558 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:22:57.426617 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.426565 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-sf295\"" Jun 09 01:22:57.426778 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.426741 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 09 01:22:57.426860 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.426817 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 09 01:22:57.427044 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.427026 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 09 01:22:57.427102 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.427090 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 09 01:22:57.440801 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.440766 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb"] Jun 09 01:22:57.522781 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.522747 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ced72e5-a97d-494c-955f-c3984f3d07fe-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c5668658c-7hfvb\" (UID: \"2ced72e5-a97d-494c-955f-c3984f3d07fe\") " pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:22:57.522936 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.522804 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2ced72e5-a97d-494c-955f-c3984f3d07fe-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c5668658c-7hfvb\" (UID: \"2ced72e5-a97d-494c-955f-c3984f3d07fe\") " pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:22:57.522936 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.522903 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4h2h\" (UniqueName: \"kubernetes.io/projected/2ced72e5-a97d-494c-955f-c3984f3d07fe-kube-api-access-g4h2h\") pod \"opendatahub-operator-controller-manager-7c5668658c-7hfvb\" (UID: \"2ced72e5-a97d-494c-955f-c3984f3d07fe\") " pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:22:57.523021 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.522948 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2ced72e5-a97d-494c-955f-c3984f3d07fe-manifests\") pod \"opendatahub-operator-controller-manager-7c5668658c-7hfvb\" (UID: \"2ced72e5-a97d-494c-955f-c3984f3d07fe\") " pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:22:57.523021 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.522966 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ced72e5-a97d-494c-955f-c3984f3d07fe-tmp\") pod \"opendatahub-operator-controller-manager-7c5668658c-7hfvb\" (UID: \"2ced72e5-a97d-494c-955f-c3984f3d07fe\") " pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:22:57.624058 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.624019 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2ced72e5-a97d-494c-955f-c3984f3d07fe-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c5668658c-7hfvb\" (UID: \"2ced72e5-a97d-494c-955f-c3984f3d07fe\") " pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:22:57.624247 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.624086 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4h2h\" (UniqueName: \"kubernetes.io/projected/2ced72e5-a97d-494c-955f-c3984f3d07fe-kube-api-access-g4h2h\") pod \"opendatahub-operator-controller-manager-7c5668658c-7hfvb\" (UID: \"2ced72e5-a97d-494c-955f-c3984f3d07fe\") " pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:22:57.624247 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.624113 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2ced72e5-a97d-494c-955f-c3984f3d07fe-manifests\") pod \"opendatahub-operator-controller-manager-7c5668658c-7hfvb\" (UID: \"2ced72e5-a97d-494c-955f-c3984f3d07fe\") " pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:22:57.624247 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.624132 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ced72e5-a97d-494c-955f-c3984f3d07fe-tmp\") pod \"opendatahub-operator-controller-manager-7c5668658c-7hfvb\" (UID: \"2ced72e5-a97d-494c-955f-c3984f3d07fe\") " pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:22:57.624247 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.624161 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ced72e5-a97d-494c-955f-c3984f3d07fe-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c5668658c-7hfvb\" (UID: \"2ced72e5-a97d-494c-955f-c3984f3d07fe\") " pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:22:57.624634 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.624603 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2ced72e5-a97d-494c-955f-c3984f3d07fe-manifests\") pod \"opendatahub-operator-controller-manager-7c5668658c-7hfvb\" (UID: \"2ced72e5-a97d-494c-955f-c3984f3d07fe\") " pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:22:57.626413 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.626383 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ced72e5-a97d-494c-955f-c3984f3d07fe-tmp\") pod \"opendatahub-operator-controller-manager-7c5668658c-7hfvb\" (UID: \"2ced72e5-a97d-494c-955f-c3984f3d07fe\") " pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:22:57.626608 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.626592 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2ced72e5-a97d-494c-955f-c3984f3d07fe-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c5668658c-7hfvb\" (UID: \"2ced72e5-a97d-494c-955f-c3984f3d07fe\") " pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:22:57.626681 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.626666 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ced72e5-a97d-494c-955f-c3984f3d07fe-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c5668658c-7hfvb\" (UID: \"2ced72e5-a97d-494c-955f-c3984f3d07fe\") " pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:22:57.632415 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.632390 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4h2h\" (UniqueName: \"kubernetes.io/projected/2ced72e5-a97d-494c-955f-c3984f3d07fe-kube-api-access-g4h2h\") pod \"opendatahub-operator-controller-manager-7c5668658c-7hfvb\" (UID: \"2ced72e5-a97d-494c-955f-c3984f3d07fe\") " pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:22:57.733417 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.733332 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:22:57.866860 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:57.866835 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb"] Jun 09 01:22:57.869015 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:22:57.868987 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ced72e5_a97d_494c_955f_c3984f3d07fe.slice/crio-3a101952ea99771bb8d51bfac9ddccaadecedded19c6709252ab83d51cae402b WatchSource:0}: Error finding container 3a101952ea99771bb8d51bfac9ddccaadecedded19c6709252ab83d51cae402b: Status 404 returned error can't find the container with id 3a101952ea99771bb8d51bfac9ddccaadecedded19c6709252ab83d51cae402b Jun 09 01:22:58.320700 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:22:58.320666 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" event={"ID":"2ced72e5-a97d-494c-955f-c3984f3d07fe","Type":"ContainerStarted","Data":"3a101952ea99771bb8d51bfac9ddccaadecedded19c6709252ab83d51cae402b"} Jun 09 01:23:01.338363 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:01.338319 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" event={"ID":"2ced72e5-a97d-494c-955f-c3984f3d07fe","Type":"ContainerStarted","Data":"7726d0375fa3b5245eede787c05364be022ee98123704a523aa6d1772e46d0ec"} Jun 09 01:23:02.344743 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:02.344694 2555 generic.go:356] "Generic (PLEG): container finished" podID="2ced72e5-a97d-494c-955f-c3984f3d07fe" containerID="7726d0375fa3b5245eede787c05364be022ee98123704a523aa6d1772e46d0ec" exitCode=0 Jun 09 01:23:02.345115 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:02.344761 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" event={"ID":"2ced72e5-a97d-494c-955f-c3984f3d07fe","Type":"ContainerDied","Data":"7726d0375fa3b5245eede787c05364be022ee98123704a523aa6d1772e46d0ec"} Jun 09 01:23:03.352030 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:03.351996 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" event={"ID":"2ced72e5-a97d-494c-955f-c3984f3d07fe","Type":"ContainerStarted","Data":"0fb2f6dfc39d4ba28c75e6ffc2f416f218361fd4e03b8ac4978c161104238441"} Jun 09 01:23:03.352421 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:03.352093 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:23:03.388404 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:03.388354 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" podStartSLOduration=2.087508607 podStartE2EDuration="6.388338813s" totalImagesPullingTime="3.242088067s" totalInitContainerRuntime="1.058742139s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:22:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:23:03.38659244 +0000 UTC m=+444.404896629" watchObservedRunningTime="2026-06-09 01:23:03.388338813 +0000 UTC m=+444.406643004" Jun 09 01:23:14.358880 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:14.358852 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c5668658c-7hfvb" Jun 09 01:23:16.423959 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:16.423927 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-tt475"] Jun 09 01:23:16.431229 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:16.431203 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-tt475" Jun 09 01:23:16.434146 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:16.434116 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-zknqp\"" Jun 09 01:23:16.435555 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:16.435505 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 09 01:23:16.436264 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:16.436241 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-tt475"] Jun 09 01:23:16.598326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:16.598287 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbzzs\" (UniqueName: \"kubernetes.io/projected/9beeea4f-dbf8-4ca9-b0d5-74c15e65f365-kube-api-access-fbzzs\") pod \"odh-model-controller-6dc94f689f-tt475\" (UID: \"9beeea4f-dbf8-4ca9-b0d5-74c15e65f365\") " pod="opendatahub/odh-model-controller-6dc94f689f-tt475" Jun 09 01:23:16.598506 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:16.598351 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9beeea4f-dbf8-4ca9-b0d5-74c15e65f365-cert\") pod \"odh-model-controller-6dc94f689f-tt475\" (UID: \"9beeea4f-dbf8-4ca9-b0d5-74c15e65f365\") " pod="opendatahub/odh-model-controller-6dc94f689f-tt475" Jun 09 01:23:16.699817 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:16.699719 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbzzs\" (UniqueName: \"kubernetes.io/projected/9beeea4f-dbf8-4ca9-b0d5-74c15e65f365-kube-api-access-fbzzs\") pod \"odh-model-controller-6dc94f689f-tt475\" (UID: \"9beeea4f-dbf8-4ca9-b0d5-74c15e65f365\") " pod="opendatahub/odh-model-controller-6dc94f689f-tt475" Jun 09 01:23:16.699817 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:16.699760 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9beeea4f-dbf8-4ca9-b0d5-74c15e65f365-cert\") pod \"odh-model-controller-6dc94f689f-tt475\" (UID: \"9beeea4f-dbf8-4ca9-b0d5-74c15e65f365\") " pod="opendatahub/odh-model-controller-6dc94f689f-tt475" Jun 09 01:23:16.700044 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:23:16.699881 2555 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 09 01:23:16.700044 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:23:16.699939 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9beeea4f-dbf8-4ca9-b0d5-74c15e65f365-cert podName:9beeea4f-dbf8-4ca9-b0d5-74c15e65f365 nodeName:}" failed. No retries permitted until 2026-06-09 01:23:17.19992255 +0000 UTC m=+458.218226740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9beeea4f-dbf8-4ca9-b0d5-74c15e65f365-cert") pod "odh-model-controller-6dc94f689f-tt475" (UID: "9beeea4f-dbf8-4ca9-b0d5-74c15e65f365") : secret "odh-model-controller-webhook-cert" not found Jun 09 01:23:16.710465 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:16.710439 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbzzs\" (UniqueName: \"kubernetes.io/projected/9beeea4f-dbf8-4ca9-b0d5-74c15e65f365-kube-api-access-fbzzs\") pod \"odh-model-controller-6dc94f689f-tt475\" (UID: \"9beeea4f-dbf8-4ca9-b0d5-74c15e65f365\") " pod="opendatahub/odh-model-controller-6dc94f689f-tt475" Jun 09 01:23:16.832392 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:16.832358 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-8jkjs"] Jun 09 01:23:16.836462 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:16.836430 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-8jkjs" Jun 09 01:23:16.839303 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:16.839279 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 09 01:23:16.845893 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:16.845866 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-8jkjs"] Jun 09 01:23:17.003167 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:17.003069 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/025ef825-c4e6-4e25-826b-d27d50dee348-tls-certs\") pod \"model-serving-api-5dd488fc48-8jkjs\" (UID: \"025ef825-c4e6-4e25-826b-d27d50dee348\") " pod="opendatahub/model-serving-api-5dd488fc48-8jkjs" Jun 09 01:23:17.003338 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:17.003218 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq5kf\" (UniqueName: \"kubernetes.io/projected/025ef825-c4e6-4e25-826b-d27d50dee348-kube-api-access-lq5kf\") pod \"model-serving-api-5dd488fc48-8jkjs\" (UID: \"025ef825-c4e6-4e25-826b-d27d50dee348\") " pod="opendatahub/model-serving-api-5dd488fc48-8jkjs" Jun 09 01:23:17.104036 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:17.104003 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq5kf\" (UniqueName: \"kubernetes.io/projected/025ef825-c4e6-4e25-826b-d27d50dee348-kube-api-access-lq5kf\") pod \"model-serving-api-5dd488fc48-8jkjs\" (UID: \"025ef825-c4e6-4e25-826b-d27d50dee348\") " pod="opendatahub/model-serving-api-5dd488fc48-8jkjs" Jun 09 01:23:17.104182 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:17.104050 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/025ef825-c4e6-4e25-826b-d27d50dee348-tls-certs\") pod \"model-serving-api-5dd488fc48-8jkjs\" (UID: \"025ef825-c4e6-4e25-826b-d27d50dee348\") " pod="opendatahub/model-serving-api-5dd488fc48-8jkjs" Jun 09 01:23:17.106428 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:17.106404 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/025ef825-c4e6-4e25-826b-d27d50dee348-tls-certs\") pod \"model-serving-api-5dd488fc48-8jkjs\" (UID: \"025ef825-c4e6-4e25-826b-d27d50dee348\") " pod="opendatahub/model-serving-api-5dd488fc48-8jkjs" Jun 09 01:23:17.112244 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:17.112214 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq5kf\" (UniqueName: \"kubernetes.io/projected/025ef825-c4e6-4e25-826b-d27d50dee348-kube-api-access-lq5kf\") pod \"model-serving-api-5dd488fc48-8jkjs\" (UID: \"025ef825-c4e6-4e25-826b-d27d50dee348\") " pod="opendatahub/model-serving-api-5dd488fc48-8jkjs" Jun 09 01:23:17.155851 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:17.155816 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-8jkjs" Jun 09 01:23:17.205270 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:17.205223 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9beeea4f-dbf8-4ca9-b0d5-74c15e65f365-cert\") pod \"odh-model-controller-6dc94f689f-tt475\" (UID: \"9beeea4f-dbf8-4ca9-b0d5-74c15e65f365\") " pod="opendatahub/odh-model-controller-6dc94f689f-tt475" Jun 09 01:23:17.208036 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:17.207998 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9beeea4f-dbf8-4ca9-b0d5-74c15e65f365-cert\") pod \"odh-model-controller-6dc94f689f-tt475\" (UID: \"9beeea4f-dbf8-4ca9-b0d5-74c15e65f365\") " pod="opendatahub/odh-model-controller-6dc94f689f-tt475" Jun 09 01:23:17.282922 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:17.282897 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-8jkjs"] Jun 09 01:23:17.284778 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:23:17.284746 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod025ef825_c4e6_4e25_826b_d27d50dee348.slice/crio-86b54006d378e88d9783441b42da7b1aa70d23cefddfdd217424ceffd20f4cc1 WatchSource:0}: Error finding container 86b54006d378e88d9783441b42da7b1aa70d23cefddfdd217424ceffd20f4cc1: Status 404 returned error can't find the container with id 86b54006d378e88d9783441b42da7b1aa70d23cefddfdd217424ceffd20f4cc1 Jun 09 01:23:17.342554 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:17.342522 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-tt475" Jun 09 01:23:17.421178 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:17.421148 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-8jkjs" event={"ID":"025ef825-c4e6-4e25-826b-d27d50dee348","Type":"ContainerStarted","Data":"86b54006d378e88d9783441b42da7b1aa70d23cefddfdd217424ceffd20f4cc1"} Jun 09 01:23:17.472231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:17.472196 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-tt475"] Jun 09 01:23:17.474244 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:23:17.474203 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9beeea4f_dbf8_4ca9_b0d5_74c15e65f365.slice/crio-83850e1e5d02b7de2578ad919de111c9933add0a87f694c5b957e9246749da11 WatchSource:0}: Error finding container 83850e1e5d02b7de2578ad919de111c9933add0a87f694c5b957e9246749da11: Status 404 returned error can't find the container with id 83850e1e5d02b7de2578ad919de111c9933add0a87f694c5b957e9246749da11 Jun 09 01:23:18.430319 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:18.430280 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-tt475" event={"ID":"9beeea4f-dbf8-4ca9-b0d5-74c15e65f365","Type":"ContainerStarted","Data":"83850e1e5d02b7de2578ad919de111c9933add0a87f694c5b957e9246749da11"} Jun 09 01:23:21.448650 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:21.448615 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-8jkjs" event={"ID":"025ef825-c4e6-4e25-826b-d27d50dee348","Type":"ContainerStarted","Data":"331a96894244cb700f454d5efe1aac646139f0d7feda099288756e9436ae4a06"} Jun 09 01:23:21.449062 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:21.448716 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-5dd488fc48-8jkjs" Jun 09 01:23:21.450300 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:21.450275 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-tt475" event={"ID":"9beeea4f-dbf8-4ca9-b0d5-74c15e65f365","Type":"ContainerStarted","Data":"ac0116021fa92f098e0ba4845875447a1bb7566eef5aad6ecdc70f1044e0d1d3"} Jun 09 01:23:21.450449 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:21.450323 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-6dc94f689f-tt475" Jun 09 01:23:21.465485 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:21.465433 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-5dd488fc48-8jkjs" podStartSLOduration=1.514143619 podStartE2EDuration="5.46541816s" totalImagesPullingTime="3.951274541s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:23:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:23:21.463946251 +0000 UTC m=+462.482250489" watchObservedRunningTime="2026-06-09 01:23:21.46541816 +0000 UTC m=+462.483722353" Jun 09 01:23:21.479394 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:21.479355 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-6dc94f689f-tt475" podStartSLOduration=1.676601389 podStartE2EDuration="5.479338464s" totalImagesPullingTime="3.802737075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:23:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:23:21.477982889 +0000 UTC m=+462.496287129" watchObservedRunningTime="2026-06-09 01:23:21.479338464 +0000 UTC m=+462.497642659" Jun 09 01:23:25.240793 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.240755 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-sln9m"] Jun 09 01:23:25.244652 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.244624 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-sln9m" Jun 09 01:23:25.247684 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.247639 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 09 01:23:25.248301 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.248061 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-fgnfl\"" Jun 09 01:23:25.253173 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.252961 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-2w2gc"] Jun 09 01:23:25.256808 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.256788 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-sln9m"] Jun 09 01:23:25.256906 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.256873 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-2w2gc" Jun 09 01:23:25.259917 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.259903 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jun 09 01:23:25.260054 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.259978 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-mnkcx\"" Jun 09 01:23:25.266202 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.266181 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-2w2gc"] Jun 09 01:23:25.278862 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.278820 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pr4f\" (UniqueName: \"kubernetes.io/projected/71b73ba0-84d3-43b2-ae1e-5493b500011e-kube-api-access-6pr4f\") pod \"kserve-controller-manager-69fc866d76-sln9m\" (UID: \"71b73ba0-84d3-43b2-ae1e-5493b500011e\") " pod="opendatahub/kserve-controller-manager-69fc866d76-sln9m" Jun 09 01:23:25.278862 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.278849 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f0cad10-da23-4ab9-9710-6cf8402e8790-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-2w2gc\" (UID: \"6f0cad10-da23-4ab9-9710-6cf8402e8790\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-2w2gc" Jun 09 01:23:25.279055 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.278957 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71b73ba0-84d3-43b2-ae1e-5493b500011e-cert\") pod \"kserve-controller-manager-69fc866d76-sln9m\" (UID: \"71b73ba0-84d3-43b2-ae1e-5493b500011e\") " pod="opendatahub/kserve-controller-manager-69fc866d76-sln9m" Jun 09 01:23:25.279055 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.279042 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7r8x\" (UniqueName: \"kubernetes.io/projected/6f0cad10-da23-4ab9-9710-6cf8402e8790-kube-api-access-z7r8x\") pod \"llmisvc-controller-manager-864bfd7bc9-2w2gc\" (UID: \"6f0cad10-da23-4ab9-9710-6cf8402e8790\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-2w2gc" Jun 09 01:23:25.380314 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.380276 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pr4f\" (UniqueName: \"kubernetes.io/projected/71b73ba0-84d3-43b2-ae1e-5493b500011e-kube-api-access-6pr4f\") pod \"kserve-controller-manager-69fc866d76-sln9m\" (UID: \"71b73ba0-84d3-43b2-ae1e-5493b500011e\") " pod="opendatahub/kserve-controller-manager-69fc866d76-sln9m" Jun 09 01:23:25.380314 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.380323 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f0cad10-da23-4ab9-9710-6cf8402e8790-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-2w2gc\" (UID: \"6f0cad10-da23-4ab9-9710-6cf8402e8790\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-2w2gc" Jun 09 01:23:25.380606 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.380408 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71b73ba0-84d3-43b2-ae1e-5493b500011e-cert\") pod \"kserve-controller-manager-69fc866d76-sln9m\" (UID: \"71b73ba0-84d3-43b2-ae1e-5493b500011e\") " pod="opendatahub/kserve-controller-manager-69fc866d76-sln9m" Jun 09 01:23:25.380606 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.380467 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7r8x\" (UniqueName: \"kubernetes.io/projected/6f0cad10-da23-4ab9-9710-6cf8402e8790-kube-api-access-z7r8x\") pod \"llmisvc-controller-manager-864bfd7bc9-2w2gc\" (UID: \"6f0cad10-da23-4ab9-9710-6cf8402e8790\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-2w2gc" Jun 09 01:23:25.380606 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:23:25.380477 2555 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jun 09 01:23:25.380606 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:23:25.380573 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f0cad10-da23-4ab9-9710-6cf8402e8790-cert podName:6f0cad10-da23-4ab9-9710-6cf8402e8790 nodeName:}" failed. No retries permitted until 2026-06-09 01:23:25.880525636 +0000 UTC m=+466.898829807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6f0cad10-da23-4ab9-9710-6cf8402e8790-cert") pod "llmisvc-controller-manager-864bfd7bc9-2w2gc" (UID: "6f0cad10-da23-4ab9-9710-6cf8402e8790") : secret "llmisvc-webhook-server-cert" not found Jun 09 01:23:25.380606 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:23:25.380574 2555 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jun 09 01:23:25.380849 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:23:25.380650 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71b73ba0-84d3-43b2-ae1e-5493b500011e-cert podName:71b73ba0-84d3-43b2-ae1e-5493b500011e nodeName:}" failed. No retries permitted until 2026-06-09 01:23:25.880623065 +0000 UTC m=+466.898927240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/71b73ba0-84d3-43b2-ae1e-5493b500011e-cert") pod "kserve-controller-manager-69fc866d76-sln9m" (UID: "71b73ba0-84d3-43b2-ae1e-5493b500011e") : secret "kserve-webhook-server-cert" not found Jun 09 01:23:25.389196 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.389168 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7r8x\" (UniqueName: \"kubernetes.io/projected/6f0cad10-da23-4ab9-9710-6cf8402e8790-kube-api-access-z7r8x\") pod \"llmisvc-controller-manager-864bfd7bc9-2w2gc\" (UID: \"6f0cad10-da23-4ab9-9710-6cf8402e8790\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-2w2gc" Jun 09 01:23:25.389503 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.389486 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pr4f\" (UniqueName: \"kubernetes.io/projected/71b73ba0-84d3-43b2-ae1e-5493b500011e-kube-api-access-6pr4f\") pod \"kserve-controller-manager-69fc866d76-sln9m\" (UID: \"71b73ba0-84d3-43b2-ae1e-5493b500011e\") " pod="opendatahub/kserve-controller-manager-69fc866d76-sln9m" Jun 09 01:23:25.884740 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.884696 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f0cad10-da23-4ab9-9710-6cf8402e8790-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-2w2gc\" (UID: \"6f0cad10-da23-4ab9-9710-6cf8402e8790\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-2w2gc" Jun 09 01:23:25.884919 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.884821 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71b73ba0-84d3-43b2-ae1e-5493b500011e-cert\") pod \"kserve-controller-manager-69fc866d76-sln9m\" (UID: \"71b73ba0-84d3-43b2-ae1e-5493b500011e\") " pod="opendatahub/kserve-controller-manager-69fc866d76-sln9m" Jun 09 01:23:25.887237 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.887201 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/71b73ba0-84d3-43b2-ae1e-5493b500011e-cert\") pod \"kserve-controller-manager-69fc866d76-sln9m\" (UID: \"71b73ba0-84d3-43b2-ae1e-5493b500011e\") " pod="opendatahub/kserve-controller-manager-69fc866d76-sln9m" Jun 09 01:23:25.887354 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:25.887272 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f0cad10-da23-4ab9-9710-6cf8402e8790-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-2w2gc\" (UID: \"6f0cad10-da23-4ab9-9710-6cf8402e8790\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-2w2gc" Jun 09 01:23:26.158436 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.158354 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-sln9m" Jun 09 01:23:26.169637 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.169536 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-2w2gc" Jun 09 01:23:26.321881 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.321850 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-sln9m"] Jun 09 01:23:26.322727 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:23:26.322701 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71b73ba0_84d3_43b2_ae1e_5493b500011e.slice/crio-b952f4b1d6d6d35cb98bd2a26d87575bee7df19ef50e3f71b5817eaee5f5f750 WatchSource:0}: Error finding container b952f4b1d6d6d35cb98bd2a26d87575bee7df19ef50e3f71b5817eaee5f5f750: Status 404 returned error can't find the container with id b952f4b1d6d6d35cb98bd2a26d87575bee7df19ef50e3f71b5817eaee5f5f750 Jun 09 01:23:26.334338 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.334301 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-2w2gc"] Jun 09 01:23:26.339359 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:23:26.339333 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6f0cad10_da23_4ab9_9710_6cf8402e8790.slice/crio-2b0561b80f1c722d84e0a97b2baf181526621d62fc4fd2664eccc137ef10b270 WatchSource:0}: Error finding container 2b0561b80f1c722d84e0a97b2baf181526621d62fc4fd2664eccc137ef10b270: Status 404 returned error can't find the container with id 2b0561b80f1c722d84e0a97b2baf181526621d62fc4fd2664eccc137ef10b270 Jun 09 01:23:26.476713 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.476594 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-sln9m" event={"ID":"71b73ba0-84d3-43b2-ae1e-5493b500011e","Type":"ContainerStarted","Data":"b952f4b1d6d6d35cb98bd2a26d87575bee7df19ef50e3f71b5817eaee5f5f750"} Jun 09 01:23:26.477758 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.477737 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-2w2gc" event={"ID":"6f0cad10-da23-4ab9-9710-6cf8402e8790","Type":"ContainerStarted","Data":"2b0561b80f1c722d84e0a97b2baf181526621d62fc4fd2664eccc137ef10b270"} Jun 09 01:23:26.707485 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.707448 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh"] Jun 09 01:23:26.711835 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.711810 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh" Jun 09 01:23:26.715697 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.715672 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 09 01:23:26.715697 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.715700 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wl4bz\"" Jun 09 01:23:26.719061 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.719032 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 09 01:23:26.719822 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.719798 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 09 01:23:26.720697 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.720678 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 09 01:23:26.728758 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.728693 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh"] Jun 09 01:23:26.733610 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.733588 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq"] Jun 09 01:23:26.737560 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.737524 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" Jun 09 01:23:26.740892 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.740875 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6rf6w\"" Jun 09 01:23:26.741281 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.741236 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 09 01:23:26.741388 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.741329 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 09 01:23:26.758521 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.758500 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq"] Jun 09 01:23:26.793264 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.793235 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk9v6\" (UniqueName: \"kubernetes.io/projected/90140303-1ad0-4667-b9cb-44640e7f2877-kube-api-access-nk9v6\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq\" (UID: \"90140303-1ad0-4667-b9cb-44640e7f2877\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" Jun 09 01:23:26.793442 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.793297 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70072497-b7f2-4a60-a5f3-29600e8953ba-tmp\") pod \"kube-auth-proxy-7c846984dd-5p7nh\" (UID: \"70072497-b7f2-4a60-a5f3-29600e8953ba\") " pod="openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh" Jun 09 01:23:26.793442 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.793389 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70072497-b7f2-4a60-a5f3-29600e8953ba-tls-certs\") pod \"kube-auth-proxy-7c846984dd-5p7nh\" (UID: \"70072497-b7f2-4a60-a5f3-29600e8953ba\") " pod="openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh" Jun 09 01:23:26.793442 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.793434 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90140303-1ad0-4667-b9cb-44640e7f2877-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq\" (UID: \"90140303-1ad0-4667-b9cb-44640e7f2877\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" Jun 09 01:23:26.793654 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.793464 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90140303-1ad0-4667-b9cb-44640e7f2877-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq\" (UID: \"90140303-1ad0-4667-b9cb-44640e7f2877\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" Jun 09 01:23:26.793654 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.793512 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb65q\" (UniqueName: \"kubernetes.io/projected/70072497-b7f2-4a60-a5f3-29600e8953ba-kube-api-access-vb65q\") pod \"kube-auth-proxy-7c846984dd-5p7nh\" (UID: \"70072497-b7f2-4a60-a5f3-29600e8953ba\") " pod="openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh" Jun 09 01:23:26.894801 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.894767 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk9v6\" (UniqueName: \"kubernetes.io/projected/90140303-1ad0-4667-b9cb-44640e7f2877-kube-api-access-nk9v6\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq\" (UID: \"90140303-1ad0-4667-b9cb-44640e7f2877\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" Jun 09 01:23:26.894964 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.894847 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70072497-b7f2-4a60-a5f3-29600e8953ba-tmp\") pod \"kube-auth-proxy-7c846984dd-5p7nh\" (UID: \"70072497-b7f2-4a60-a5f3-29600e8953ba\") " pod="openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh" Jun 09 01:23:26.894964 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.894895 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70072497-b7f2-4a60-a5f3-29600e8953ba-tls-certs\") pod \"kube-auth-proxy-7c846984dd-5p7nh\" (UID: \"70072497-b7f2-4a60-a5f3-29600e8953ba\") " pod="openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh" Jun 09 01:23:26.894964 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.894927 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90140303-1ad0-4667-b9cb-44640e7f2877-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq\" (UID: \"90140303-1ad0-4667-b9cb-44640e7f2877\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" Jun 09 01:23:26.895110 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.894969 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90140303-1ad0-4667-b9cb-44640e7f2877-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq\" (UID: \"90140303-1ad0-4667-b9cb-44640e7f2877\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" Jun 09 01:23:26.895110 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.895021 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb65q\" (UniqueName: \"kubernetes.io/projected/70072497-b7f2-4a60-a5f3-29600e8953ba-kube-api-access-vb65q\") pod \"kube-auth-proxy-7c846984dd-5p7nh\" (UID: \"70072497-b7f2-4a60-a5f3-29600e8953ba\") " pod="openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh" Jun 09 01:23:26.895110 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:23:26.895050 2555 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jun 09 01:23:26.895255 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:23:26.895138 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/70072497-b7f2-4a60-a5f3-29600e8953ba-tls-certs podName:70072497-b7f2-4a60-a5f3-29600e8953ba nodeName:}" failed. No retries permitted until 2026-06-09 01:23:27.39511751 +0000 UTC m=+468.413421687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/70072497-b7f2-4a60-a5f3-29600e8953ba-tls-certs") pod "kube-auth-proxy-7c846984dd-5p7nh" (UID: "70072497-b7f2-4a60-a5f3-29600e8953ba") : secret "kube-auth-proxy-tls" not found Jun 09 01:23:26.895321 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.895268 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90140303-1ad0-4667-b9cb-44640e7f2877-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq\" (UID: \"90140303-1ad0-4667-b9cb-44640e7f2877\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" Jun 09 01:23:26.895375 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.895343 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90140303-1ad0-4667-b9cb-44640e7f2877-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq\" (UID: \"90140303-1ad0-4667-b9cb-44640e7f2877\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" Jun 09 01:23:26.897127 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.897106 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70072497-b7f2-4a60-a5f3-29600e8953ba-tmp\") pod \"kube-auth-proxy-7c846984dd-5p7nh\" (UID: \"70072497-b7f2-4a60-a5f3-29600e8953ba\") " pod="openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh" Jun 09 01:23:26.903680 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.903653 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb65q\" (UniqueName: \"kubernetes.io/projected/70072497-b7f2-4a60-a5f3-29600e8953ba-kube-api-access-vb65q\") pod \"kube-auth-proxy-7c846984dd-5p7nh\" (UID: \"70072497-b7f2-4a60-a5f3-29600e8953ba\") " pod="openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh" Jun 09 01:23:26.903771 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:26.903658 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk9v6\" (UniqueName: \"kubernetes.io/projected/90140303-1ad0-4667-b9cb-44640e7f2877-kube-api-access-nk9v6\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq\" (UID: \"90140303-1ad0-4667-b9cb-44640e7f2877\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" Jun 09 01:23:27.050196 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:27.050141 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" Jun 09 01:23:27.387827 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:27.387782 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq"] Jun 09 01:23:27.401198 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:27.401167 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70072497-b7f2-4a60-a5f3-29600e8953ba-tls-certs\") pod \"kube-auth-proxy-7c846984dd-5p7nh\" (UID: \"70072497-b7f2-4a60-a5f3-29600e8953ba\") " pod="openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh" Jun 09 01:23:27.403492 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:27.403467 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70072497-b7f2-4a60-a5f3-29600e8953ba-tls-certs\") pod \"kube-auth-proxy-7c846984dd-5p7nh\" (UID: \"70072497-b7f2-4a60-a5f3-29600e8953ba\") " pod="openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh" Jun 09 01:23:27.488870 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:23:27.488833 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90140303_1ad0_4667_b9cb_44640e7f2877.slice/crio-912bccdb3fc68be50fe56960f4797ec7fee5a5243687690bf56c22a1074908b0 WatchSource:0}: Error finding container 912bccdb3fc68be50fe56960f4797ec7fee5a5243687690bf56c22a1074908b0: Status 404 returned error can't find the container with id 912bccdb3fc68be50fe56960f4797ec7fee5a5243687690bf56c22a1074908b0 Jun 09 01:23:27.623063 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:27.623035 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh" Jun 09 01:23:27.841276 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:27.841250 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh"] Jun 09 01:23:27.846488 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:23:27.846440 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70072497_b7f2_4a60_a5f3_29600e8953ba.slice/crio-97439f90314de45a71ecac8d6ce041ebd2037719fb69a186b1dede19dfbf84e6 WatchSource:0}: Error finding container 97439f90314de45a71ecac8d6ce041ebd2037719fb69a186b1dede19dfbf84e6: Status 404 returned error can't find the container with id 97439f90314de45a71ecac8d6ce041ebd2037719fb69a186b1dede19dfbf84e6 Jun 09 01:23:28.494633 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:28.494591 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh" event={"ID":"70072497-b7f2-4a60-a5f3-29600e8953ba","Type":"ContainerStarted","Data":"97439f90314de45a71ecac8d6ce041ebd2037719fb69a186b1dede19dfbf84e6"} Jun 09 01:23:28.496259 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:28.496232 2555 generic.go:356] "Generic (PLEG): container finished" podID="90140303-1ad0-4667-b9cb-44640e7f2877" containerID="cbe0b1dd8ef42d87c919df20136bd26cc2baed219ac4b869371ca48d569c6e44" exitCode=0 Jun 09 01:23:28.496388 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:28.496284 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" event={"ID":"90140303-1ad0-4667-b9cb-44640e7f2877","Type":"ContainerDied","Data":"cbe0b1dd8ef42d87c919df20136bd26cc2baed219ac4b869371ca48d569c6e44"} Jun 09 01:23:28.496388 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:28.496319 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" event={"ID":"90140303-1ad0-4667-b9cb-44640e7f2877","Type":"ContainerStarted","Data":"912bccdb3fc68be50fe56960f4797ec7fee5a5243687690bf56c22a1074908b0"} Jun 09 01:23:30.510117 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:30.510075 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-2w2gc" event={"ID":"6f0cad10-da23-4ab9-9710-6cf8402e8790","Type":"ContainerStarted","Data":"f6b88ffdd8fe393c8306261491212b7c11128b6b701b19819021bd33b9ca8352"} Jun 09 01:23:30.510605 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:30.510276 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-2w2gc" Jun 09 01:23:30.512133 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:30.512099 2555 generic.go:356] "Generic (PLEG): container finished" podID="90140303-1ad0-4667-b9cb-44640e7f2877" containerID="10aa117895e70b03c0e6b1801b30970bffbf4968db9bc02d32b1689926d207bf" exitCode=0 Jun 09 01:23:30.512266 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:30.512191 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" event={"ID":"90140303-1ad0-4667-b9cb-44640e7f2877","Type":"ContainerDied","Data":"10aa117895e70b03c0e6b1801b30970bffbf4968db9bc02d32b1689926d207bf"} Jun 09 01:23:30.513761 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:30.513738 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-sln9m" event={"ID":"71b73ba0-84d3-43b2-ae1e-5493b500011e","Type":"ContainerStarted","Data":"985966ca449b7c00e1c346696c0b579088f816fea436b6ac9f38ba77c1f8c4e3"} Jun 09 01:23:30.513877 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:30.513863 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-69fc866d76-sln9m" Jun 09 01:23:30.529812 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:30.529767 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-2w2gc" podStartSLOduration=2.307663661 podStartE2EDuration="5.529754075s" totalImagesPullingTime="3.222090414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:23:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:23:30.527669391 +0000 UTC m=+471.545973584" watchObservedRunningTime="2026-06-09 01:23:30.529754075 +0000 UTC m=+471.548058278" Jun 09 01:23:30.553426 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:30.553371 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-69fc866d76-sln9m" podStartSLOduration=2.319812718 podStartE2EDuration="5.553348331s" totalImagesPullingTime="3.233535613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:23:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:23:30.551690025 +0000 UTC m=+471.569994217" watchObservedRunningTime="2026-06-09 01:23:30.553348331 +0000 UTC m=+471.571652524" Jun 09 01:23:31.520341 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:31.520307 2555 generic.go:356] "Generic (PLEG): container finished" podID="90140303-1ad0-4667-b9cb-44640e7f2877" containerID="da6b569f4de3ec98c6432ac7266f8dd318218ec63daeb7679e94e55d7fcff2dd" exitCode=0 Jun 09 01:23:31.520792 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:31.520392 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" event={"ID":"90140303-1ad0-4667-b9cb-44640e7f2877","Type":"ContainerDied","Data":"da6b569f4de3ec98c6432ac7266f8dd318218ec63daeb7679e94e55d7fcff2dd"} Jun 09 01:23:31.521972 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:31.521945 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh" event={"ID":"70072497-b7f2-4a60-a5f3-29600e8953ba","Type":"ContainerStarted","Data":"3c51cedc14ad73c4be624683342ef8d39af59673ea9bfcee63882837e3813bbe"} Jun 09 01:23:31.566589 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:31.566519 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c846984dd-5p7nh" podStartSLOduration=2.531092791 podStartE2EDuration="5.566504841s" totalImagesPullingTime="3.03541205s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:23:31.565264072 +0000 UTC m=+472.583568271" watchObservedRunningTime="2026-06-09 01:23:31.566504841 +0000 UTC m=+472.584809058" Jun 09 01:23:32.457768 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:32.457733 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-6dc94f689f-tt475" Jun 09 01:23:32.460512 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:32.460490 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-5dd488fc48-8jkjs" Jun 09 01:23:32.660911 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:32.660888 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" Jun 09 01:23:32.751379 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:32.751282 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nk9v6\" (UniqueName: \"kubernetes.io/projected/90140303-1ad0-4667-b9cb-44640e7f2877-kube-api-access-nk9v6\") pod \"90140303-1ad0-4667-b9cb-44640e7f2877\" (UID: \"90140303-1ad0-4667-b9cb-44640e7f2877\") " Jun 09 01:23:32.751379 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:32.751343 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90140303-1ad0-4667-b9cb-44640e7f2877-bundle\") pod \"90140303-1ad0-4667-b9cb-44640e7f2877\" (UID: \"90140303-1ad0-4667-b9cb-44640e7f2877\") " Jun 09 01:23:32.751647 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:32.751471 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90140303-1ad0-4667-b9cb-44640e7f2877-util\") pod \"90140303-1ad0-4667-b9cb-44640e7f2877\" (UID: \"90140303-1ad0-4667-b9cb-44640e7f2877\") " Jun 09 01:23:32.752247 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:32.752219 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90140303-1ad0-4667-b9cb-44640e7f2877-bundle" (OuterVolumeSpecName: "bundle") pod "90140303-1ad0-4667-b9cb-44640e7f2877" (UID: "90140303-1ad0-4667-b9cb-44640e7f2877"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:23:32.753526 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:32.753497 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90140303-1ad0-4667-b9cb-44640e7f2877-kube-api-access-nk9v6" (OuterVolumeSpecName: "kube-api-access-nk9v6") pod "90140303-1ad0-4667-b9cb-44640e7f2877" (UID: "90140303-1ad0-4667-b9cb-44640e7f2877"). InnerVolumeSpecName "kube-api-access-nk9v6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:23:32.756525 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:32.756505 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90140303-1ad0-4667-b9cb-44640e7f2877-util" (OuterVolumeSpecName: "util") pod "90140303-1ad0-4667-b9cb-44640e7f2877" (UID: "90140303-1ad0-4667-b9cb-44640e7f2877"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:23:32.852370 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:32.852332 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90140303-1ad0-4667-b9cb-44640e7f2877-util\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:23:32.852370 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:32.852365 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nk9v6\" (UniqueName: \"kubernetes.io/projected/90140303-1ad0-4667-b9cb-44640e7f2877-kube-api-access-nk9v6\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:23:32.852370 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:32.852375 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90140303-1ad0-4667-b9cb-44640e7f2877-bundle\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:23:33.534372 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:33.534341 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" event={"ID":"90140303-1ad0-4667-b9cb-44640e7f2877","Type":"ContainerDied","Data":"912bccdb3fc68be50fe56960f4797ec7fee5a5243687690bf56c22a1074908b0"} Jun 09 01:23:33.534372 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:33.534374 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06fz5fq" Jun 09 01:23:33.534575 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:33.534387 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="912bccdb3fc68be50fe56960f4797ec7fee5a5243687690bf56c22a1074908b0" Jun 09 01:23:40.778090 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.778058 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c"] Jun 09 01:23:40.778450 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.778420 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90140303-1ad0-4667-b9cb-44640e7f2877" containerName="extract" Jun 09 01:23:40.778450 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.778431 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="90140303-1ad0-4667-b9cb-44640e7f2877" containerName="extract" Jun 09 01:23:40.778450 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.778442 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90140303-1ad0-4667-b9cb-44640e7f2877" containerName="util" Jun 09 01:23:40.778450 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.778448 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="90140303-1ad0-4667-b9cb-44640e7f2877" containerName="util" Jun 09 01:23:40.778592 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.778465 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90140303-1ad0-4667-b9cb-44640e7f2877" containerName="pull" Jun 09 01:23:40.778592 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.778471 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="90140303-1ad0-4667-b9cb-44640e7f2877" containerName="pull" Jun 09 01:23:40.778592 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.778527 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="90140303-1ad0-4667-b9cb-44640e7f2877" containerName="extract" Jun 09 01:23:40.782200 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.782180 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" Jun 09 01:23:40.787704 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.787681 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 09 01:23:40.788511 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.788487 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6rf6w\"" Jun 09 01:23:40.788511 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.788507 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 09 01:23:40.807305 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.807272 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c"] Jun 09 01:23:40.827406 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.827372 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c\" (UID: \"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" Jun 09 01:23:40.827406 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.827404 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c\" (UID: \"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" Jun 09 01:23:40.827695 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.827465 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grcfg\" (UniqueName: \"kubernetes.io/projected/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-kube-api-access-grcfg\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c\" (UID: \"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" Jun 09 01:23:40.928310 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.928280 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c\" (UID: \"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" Jun 09 01:23:40.928310 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.928311 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c\" (UID: \"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" Jun 09 01:23:40.928690 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.928357 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grcfg\" (UniqueName: \"kubernetes.io/projected/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-kube-api-access-grcfg\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c\" (UID: \"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" Jun 09 01:23:40.928832 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.928811 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c\" (UID: \"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" Jun 09 01:23:40.928946 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.928836 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c\" (UID: \"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" Jun 09 01:23:40.941658 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:40.941617 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grcfg\" (UniqueName: \"kubernetes.io/projected/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-kube-api-access-grcfg\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c\" (UID: \"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" Jun 09 01:23:41.093264 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:41.093174 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" Jun 09 01:23:41.229432 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:41.229407 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c"] Jun 09 01:23:41.230173 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:23:41.230140 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffa18c6e_f8b4_4ebf_9ec0_c20183fd8b93.slice/crio-11cd93388812ae5486814a80ddda29c8faedf507492cf84c8aa7343bedae576b WatchSource:0}: Error finding container 11cd93388812ae5486814a80ddda29c8faedf507492cf84c8aa7343bedae576b: Status 404 returned error can't find the container with id 11cd93388812ae5486814a80ddda29c8faedf507492cf84c8aa7343bedae576b Jun 09 01:23:41.576051 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:41.576017 2555 generic.go:356] "Generic (PLEG): container finished" podID="ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93" containerID="6b7e97e9a363ec70d7bf5dc4c5604a2e7215a161db631a37881a847b59a5fec0" exitCode=0 Jun 09 01:23:41.576226 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:41.576059 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" event={"ID":"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93","Type":"ContainerDied","Data":"6b7e97e9a363ec70d7bf5dc4c5604a2e7215a161db631a37881a847b59a5fec0"} Jun 09 01:23:41.576226 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:41.576117 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" event={"ID":"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93","Type":"ContainerStarted","Data":"11cd93388812ae5486814a80ddda29c8faedf507492cf84c8aa7343bedae576b"} Jun 09 01:23:42.065130 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:42.065097 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-67bg9"] Jun 09 01:23:42.067688 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:42.067667 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-67bg9" Jun 09 01:23:42.071018 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:42.070998 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Jun 09 01:23:42.071097 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:42.071037 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Jun 09 01:23:42.071097 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:42.071054 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-k44vm\"" Jun 09 01:23:42.080395 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:42.080374 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-67bg9"] Jun 09 01:23:42.140539 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:42.140503 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k572q\" (UniqueName: \"kubernetes.io/projected/a2caae4b-a4fc-4fad-b165-dd23cb91ce11-kube-api-access-k572q\") pod \"servicemesh-operator3-57f65f65fb-67bg9\" (UID: \"a2caae4b-a4fc-4fad-b165-dd23cb91ce11\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-67bg9" Jun 09 01:23:42.140722 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:42.140607 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/a2caae4b-a4fc-4fad-b165-dd23cb91ce11-operator-config\") pod \"servicemesh-operator3-57f65f65fb-67bg9\" (UID: \"a2caae4b-a4fc-4fad-b165-dd23cb91ce11\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-67bg9" Jun 09 01:23:42.242075 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:42.241999 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k572q\" (UniqueName: \"kubernetes.io/projected/a2caae4b-a4fc-4fad-b165-dd23cb91ce11-kube-api-access-k572q\") pod \"servicemesh-operator3-57f65f65fb-67bg9\" (UID: \"a2caae4b-a4fc-4fad-b165-dd23cb91ce11\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-67bg9" Jun 09 01:23:42.242075 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:42.242051 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/a2caae4b-a4fc-4fad-b165-dd23cb91ce11-operator-config\") pod \"servicemesh-operator3-57f65f65fb-67bg9\" (UID: \"a2caae4b-a4fc-4fad-b165-dd23cb91ce11\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-67bg9" Jun 09 01:23:42.244367 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:42.244343 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/a2caae4b-a4fc-4fad-b165-dd23cb91ce11-operator-config\") pod \"servicemesh-operator3-57f65f65fb-67bg9\" (UID: \"a2caae4b-a4fc-4fad-b165-dd23cb91ce11\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-67bg9" Jun 09 01:23:42.259145 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:42.259114 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k572q\" (UniqueName: \"kubernetes.io/projected/a2caae4b-a4fc-4fad-b165-dd23cb91ce11-kube-api-access-k572q\") pod \"servicemesh-operator3-57f65f65fb-67bg9\" (UID: \"a2caae4b-a4fc-4fad-b165-dd23cb91ce11\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-67bg9" Jun 09 01:23:42.376812 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:42.376777 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-67bg9" Jun 09 01:23:42.510628 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:42.510603 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-67bg9"] Jun 09 01:23:42.512438 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:23:42.512414 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2caae4b_a4fc_4fad_b165_dd23cb91ce11.slice/crio-bb2d8dcdec7e0caf7eebaa3e01d25a505140e6d92e8aac3dc5f80e36f1ac6f23 WatchSource:0}: Error finding container bb2d8dcdec7e0caf7eebaa3e01d25a505140e6d92e8aac3dc5f80e36f1ac6f23: Status 404 returned error can't find the container with id bb2d8dcdec7e0caf7eebaa3e01d25a505140e6d92e8aac3dc5f80e36f1ac6f23 Jun 09 01:23:42.583107 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:42.583072 2555 generic.go:356] "Generic (PLEG): container finished" podID="ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93" containerID="334154b0d43e57d9b0727531f852a346a9b24d1b055624ca0ec28b64186391c3" exitCode=0 Jun 09 01:23:42.583265 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:42.583156 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" event={"ID":"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93","Type":"ContainerDied","Data":"334154b0d43e57d9b0727531f852a346a9b24d1b055624ca0ec28b64186391c3"} Jun 09 01:23:42.584873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:42.584853 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-67bg9" event={"ID":"a2caae4b-a4fc-4fad-b165-dd23cb91ce11","Type":"ContainerStarted","Data":"bb2d8dcdec7e0caf7eebaa3e01d25a505140e6d92e8aac3dc5f80e36f1ac6f23"} Jun 09 01:23:43.158515 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:43.158432 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-98cf8ddb4-g7ccn"] Jun 09 01:23:43.161180 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:43.161158 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-98cf8ddb4-g7ccn" Jun 09 01:23:43.164583 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:43.164539 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 09 01:23:43.164680 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:43.164570 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-c4zw6\"" Jun 09 01:23:43.172714 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:43.172687 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-98cf8ddb4-g7ccn"] Jun 09 01:23:43.252284 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:43.252240 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2ckx\" (UniqueName: \"kubernetes.io/projected/97ada88d-bc16-4fb9-98c8-0907e2ced152-kube-api-access-w2ckx\") pod \"dashboard-redirect-98cf8ddb4-g7ccn\" (UID: \"97ada88d-bc16-4fb9-98c8-0907e2ced152\") " pod="opendatahub/dashboard-redirect-98cf8ddb4-g7ccn" Jun 09 01:23:43.252284 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:43.252283 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/97ada88d-bc16-4fb9-98c8-0907e2ced152-redirect-config\") pod \"dashboard-redirect-98cf8ddb4-g7ccn\" (UID: \"97ada88d-bc16-4fb9-98c8-0907e2ced152\") " pod="opendatahub/dashboard-redirect-98cf8ddb4-g7ccn" Jun 09 01:23:43.353210 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:43.353166 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2ckx\" (UniqueName: \"kubernetes.io/projected/97ada88d-bc16-4fb9-98c8-0907e2ced152-kube-api-access-w2ckx\") pod \"dashboard-redirect-98cf8ddb4-g7ccn\" (UID: \"97ada88d-bc16-4fb9-98c8-0907e2ced152\") " pod="opendatahub/dashboard-redirect-98cf8ddb4-g7ccn" Jun 09 01:23:43.353397 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:43.353219 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/97ada88d-bc16-4fb9-98c8-0907e2ced152-redirect-config\") pod \"dashboard-redirect-98cf8ddb4-g7ccn\" (UID: \"97ada88d-bc16-4fb9-98c8-0907e2ced152\") " pod="opendatahub/dashboard-redirect-98cf8ddb4-g7ccn" Jun 09 01:23:43.354007 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:43.353976 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/97ada88d-bc16-4fb9-98c8-0907e2ced152-redirect-config\") pod \"dashboard-redirect-98cf8ddb4-g7ccn\" (UID: \"97ada88d-bc16-4fb9-98c8-0907e2ced152\") " pod="opendatahub/dashboard-redirect-98cf8ddb4-g7ccn" Jun 09 01:23:43.369359 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:43.369337 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2ckx\" (UniqueName: \"kubernetes.io/projected/97ada88d-bc16-4fb9-98c8-0907e2ced152-kube-api-access-w2ckx\") pod \"dashboard-redirect-98cf8ddb4-g7ccn\" (UID: \"97ada88d-bc16-4fb9-98c8-0907e2ced152\") " pod="opendatahub/dashboard-redirect-98cf8ddb4-g7ccn" Jun 09 01:23:43.473163 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:43.473064 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-98cf8ddb4-g7ccn" Jun 09 01:23:43.593589 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:43.593533 2555 generic.go:356] "Generic (PLEG): container finished" podID="ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93" containerID="f38c40510884a789a8044c9122ffafaf2f52f6608669f375527ac2302acd9df7" exitCode=0 Jun 09 01:23:43.593589 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:43.593573 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" event={"ID":"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93","Type":"ContainerDied","Data":"f38c40510884a789a8044c9122ffafaf2f52f6608669f375527ac2302acd9df7"} Jun 09 01:23:43.611956 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:43.611931 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-98cf8ddb4-g7ccn"] Jun 09 01:23:43.613675 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:23:43.613651 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97ada88d_bc16_4fb9_98c8_0907e2ced152.slice/crio-204c87fd5343804680d232ff379e29bfae77aca0b3d795f3d00030501a6b5a67 WatchSource:0}: Error finding container 204c87fd5343804680d232ff379e29bfae77aca0b3d795f3d00030501a6b5a67: Status 404 returned error can't find the container with id 204c87fd5343804680d232ff379e29bfae77aca0b3d795f3d00030501a6b5a67 Jun 09 01:23:44.604517 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:44.604439 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-98cf8ddb4-g7ccn" event={"ID":"97ada88d-bc16-4fb9-98c8-0907e2ced152","Type":"ContainerStarted","Data":"204c87fd5343804680d232ff379e29bfae77aca0b3d795f3d00030501a6b5a67"} Jun 09 01:23:45.602094 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:45.602068 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" Jun 09 01:23:45.611709 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:45.611676 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" event={"ID":"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93","Type":"ContainerDied","Data":"11cd93388812ae5486814a80ddda29c8faedf507492cf84c8aa7343bedae576b"} Jun 09 01:23:45.612088 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:45.611735 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="11cd93388812ae5486814a80ddda29c8faedf507492cf84c8aa7343bedae576b" Jun 09 01:23:45.612088 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:45.611740 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762a2qm8c" Jun 09 01:23:45.677948 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:45.677918 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-util\") pod \"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93\" (UID: \"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93\") " Jun 09 01:23:45.678133 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:45.678005 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-bundle\") pod \"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93\" (UID: \"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93\") " Jun 09 01:23:45.678133 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:45.678031 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grcfg\" (UniqueName: \"kubernetes.io/projected/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-kube-api-access-grcfg\") pod \"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93\" (UID: \"ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93\") " Jun 09 01:23:45.679030 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:45.678965 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-bundle" (OuterVolumeSpecName: "bundle") pod "ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93" (UID: "ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:23:45.680477 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:45.680447 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-kube-api-access-grcfg" (OuterVolumeSpecName: "kube-api-access-grcfg") pod "ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93" (UID: "ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93"). InnerVolumeSpecName "kube-api-access-grcfg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:23:45.685372 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:45.685327 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-util" (OuterVolumeSpecName: "util") pod "ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93" (UID: "ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:23:45.778994 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:45.778953 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-bundle\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:23:45.778994 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:45.778983 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grcfg\" (UniqueName: \"kubernetes.io/projected/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-kube-api-access-grcfg\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:23:45.778994 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:45.778996 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93-util\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:23:46.620224 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:46.620184 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-98cf8ddb4-g7ccn" event={"ID":"97ada88d-bc16-4fb9-98c8-0907e2ced152","Type":"ContainerStarted","Data":"7cd6873029947d7fb01ce158ab748017056bc68580f77b5ce64af45b2b67aade"} Jun 09 01:23:46.622412 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:46.622350 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-67bg9" event={"ID":"a2caae4b-a4fc-4fad-b165-dd23cb91ce11","Type":"ContainerStarted","Data":"1855ab937cbfa5b6e684bc4dfe95f3198be2b40a4f4e9f491d99cfac2b66e29f"} Jun 09 01:23:46.622412 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:46.622406 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-67bg9" Jun 09 01:23:46.640656 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:46.640514 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-98cf8ddb4-g7ccn" podStartSLOduration=0.822348246 podStartE2EDuration="3.640493547s" totalImagesPullingTime="2.818145301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:23:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:23:46.637522347 +0000 UTC m=+487.655826540" watchObservedRunningTime="2026-06-09 01:23:46.640493547 +0000 UTC m=+487.658797740" Jun 09 01:23:46.659384 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:46.659325 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-57f65f65fb-67bg9" podStartSLOduration=0.800075657 podStartE2EDuration="4.659279279s" totalImagesPullingTime="3.859203622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:23:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:23:46.658850688 +0000 UTC m=+487.677154901" watchObservedRunningTime="2026-06-09 01:23:46.659279279 +0000 UTC m=+487.677583474" Jun 09 01:23:57.022332 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.022294 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns"] Jun 09 01:23:57.022839 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.022823 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93" containerName="util" Jun 09 01:23:57.022886 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.022842 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93" containerName="util" Jun 09 01:23:57.022886 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.022864 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93" containerName="pull" Jun 09 01:23:57.022886 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.022872 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93" containerName="pull" Jun 09 01:23:57.022976 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.022890 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93" containerName="extract" Jun 09 01:23:57.022976 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.022899 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93" containerName="extract" Jun 09 01:23:57.023035 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.023011 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffa18c6e-f8b4-4ebf-9ec0-c20183fd8b93" containerName="extract" Jun 09 01:23:57.029159 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.029137 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.032297 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.032266 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 09 01:23:57.032297 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.032270 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 09 01:23:57.032502 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.032317 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 09 01:23:57.033604 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.033428 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 09 01:23:57.033604 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.033533 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-v98bv\"" Jun 09 01:23:57.036752 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.036723 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns"] Jun 09 01:23:57.189645 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.189615 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/96dfe581-1560-44e0-83cb-105860269997-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.189833 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.189652 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/96dfe581-1560-44e0-83cb-105860269997-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.189833 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.189687 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gcq7\" (UniqueName: \"kubernetes.io/projected/96dfe581-1560-44e0-83cb-105860269997-kube-api-access-2gcq7\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.189833 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.189709 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96dfe581-1560-44e0-83cb-105860269997-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.189833 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.189743 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/96dfe581-1560-44e0-83cb-105860269997-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.189833 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.189812 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/96dfe581-1560-44e0-83cb-105860269997-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.190012 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.189858 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/96dfe581-1560-44e0-83cb-105860269997-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.291168 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.291072 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/96dfe581-1560-44e0-83cb-105860269997-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.291168 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.291115 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/96dfe581-1560-44e0-83cb-105860269997-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.291168 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.291137 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/96dfe581-1560-44e0-83cb-105860269997-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.291446 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.291193 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/96dfe581-1560-44e0-83cb-105860269997-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.291446 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.291213 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/96dfe581-1560-44e0-83cb-105860269997-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.291446 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.291242 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gcq7\" (UniqueName: \"kubernetes.io/projected/96dfe581-1560-44e0-83cb-105860269997-kube-api-access-2gcq7\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.291446 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.291271 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96dfe581-1560-44e0-83cb-105860269997-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.291975 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.291948 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/96dfe581-1560-44e0-83cb-105860269997-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.293679 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.293632 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/96dfe581-1560-44e0-83cb-105860269997-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.293960 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.293927 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/96dfe581-1560-44e0-83cb-105860269997-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.294065 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.293971 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/96dfe581-1560-44e0-83cb-105860269997-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.294065 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.294014 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/96dfe581-1560-44e0-83cb-105860269997-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.299456 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.299432 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gcq7\" (UniqueName: \"kubernetes.io/projected/96dfe581-1560-44e0-83cb-105860269997-kube-api-access-2gcq7\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.299805 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.299789 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96dfe581-1560-44e0-83cb-105860269997-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-vfdns\" (UID: \"96dfe581-1560-44e0-83cb-105860269997\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.341171 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.341135 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:23:57.482888 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.482856 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns"] Jun 09 01:23:57.484340 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:23:57.484307 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96dfe581_1560_44e0_83cb_105860269997.slice/crio-835186ad4e85992693c5c00d320561b6321247700004f6c560230f1d3ab4ba21 WatchSource:0}: Error finding container 835186ad4e85992693c5c00d320561b6321247700004f6c560230f1d3ab4ba21: Status 404 returned error can't find the container with id 835186ad4e85992693c5c00d320561b6321247700004f6c560230f1d3ab4ba21 Jun 09 01:23:57.630156 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.630079 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-67bg9" Jun 09 01:23:57.684001 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:23:57.683958 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" event={"ID":"96dfe581-1560-44e0-83cb-105860269997","Type":"ContainerStarted","Data":"835186ad4e85992693c5c00d320561b6321247700004f6c560230f1d3ab4ba21"} Jun 09 01:24:01.524011 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:01.523978 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-2w2gc" Jun 09 01:24:01.527075 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:01.527053 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-69fc866d76-sln9m" Jun 09 01:24:03.037736 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:03.037687 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 09 01:24:03.037981 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:03.037772 2555 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jun 09 01:24:03.719494 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:03.719448 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" event={"ID":"96dfe581-1560-44e0-83cb-105860269997","Type":"ContainerStarted","Data":"a7b68a839f211846d9733e51c24b1b0f2eaa17fb9e61a7c8ec661145b1bbc4ca"} Jun 09 01:24:03.719721 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:03.719562 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:24:03.741623 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:03.741569 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" podStartSLOduration=2.190642983 podStartE2EDuration="7.741538829s" totalImagesPullingTime="5.550895846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:23:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:24:03.739417027 +0000 UTC m=+504.757721220" watchObservedRunningTime="2026-06-09 01:24:03.741538829 +0000 UTC m=+504.759843020" Jun 09 01:24:04.726751 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:04.726716 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-vfdns" Jun 09 01:24:33.358184 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.358141 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2"] Jun 09 01:24:33.369636 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.369606 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2"] Jun 09 01:24:33.369790 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.369715 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" Jun 09 01:24:33.372539 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.372516 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 09 01:24:33.374020 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.374001 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 09 01:24:33.374149 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.374018 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-f84hw\"" Jun 09 01:24:33.524537 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.524496 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a52df31-fe8a-447f-93a8-d1409e9ab36e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2\" (UID: \"2a52df31-fe8a-447f-93a8-d1409e9ab36e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" Jun 09 01:24:33.524762 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.524735 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a52df31-fe8a-447f-93a8-d1409e9ab36e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2\" (UID: \"2a52df31-fe8a-447f-93a8-d1409e9ab36e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" Jun 09 01:24:33.524825 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.524807 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24t9z\" (UniqueName: \"kubernetes.io/projected/2a52df31-fe8a-447f-93a8-d1409e9ab36e-kube-api-access-24t9z\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2\" (UID: \"2a52df31-fe8a-447f-93a8-d1409e9ab36e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" Jun 09 01:24:33.625897 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.625795 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a52df31-fe8a-447f-93a8-d1409e9ab36e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2\" (UID: \"2a52df31-fe8a-447f-93a8-d1409e9ab36e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" Jun 09 01:24:33.625897 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.625855 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24t9z\" (UniqueName: \"kubernetes.io/projected/2a52df31-fe8a-447f-93a8-d1409e9ab36e-kube-api-access-24t9z\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2\" (UID: \"2a52df31-fe8a-447f-93a8-d1409e9ab36e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" Jun 09 01:24:33.625897 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.625901 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a52df31-fe8a-447f-93a8-d1409e9ab36e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2\" (UID: \"2a52df31-fe8a-447f-93a8-d1409e9ab36e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" Jun 09 01:24:33.626347 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.626325 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a52df31-fe8a-447f-93a8-d1409e9ab36e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2\" (UID: \"2a52df31-fe8a-447f-93a8-d1409e9ab36e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" Jun 09 01:24:33.626440 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.626352 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a52df31-fe8a-447f-93a8-d1409e9ab36e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2\" (UID: \"2a52df31-fe8a-447f-93a8-d1409e9ab36e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" Jun 09 01:24:33.633821 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.633796 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24t9z\" (UniqueName: \"kubernetes.io/projected/2a52df31-fe8a-447f-93a8-d1409e9ab36e-kube-api-access-24t9z\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2\" (UID: \"2a52df31-fe8a-447f-93a8-d1409e9ab36e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" Jun 09 01:24:33.680786 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.680732 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" Jun 09 01:24:33.812908 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.812877 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2"] Jun 09 01:24:33.814431 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:24:33.814403 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a52df31_fe8a_447f_93a8_d1409e9ab36e.slice/crio-973586ad34a43b9a2167eb4adc33c4dc0872952ab7a70ab7053d71a88d3e887a WatchSource:0}: Error finding container 973586ad34a43b9a2167eb4adc33c4dc0872952ab7a70ab7053d71a88d3e887a: Status 404 returned error can't find the container with id 973586ad34a43b9a2167eb4adc33c4dc0872952ab7a70ab7053d71a88d3e887a Jun 09 01:24:33.879089 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:33.879034 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" event={"ID":"2a52df31-fe8a-447f-93a8-d1409e9ab36e","Type":"ContainerStarted","Data":"973586ad34a43b9a2167eb4adc33c4dc0872952ab7a70ab7053d71a88d3e887a"} Jun 09 01:24:34.154943 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.154865 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5"] Jun 09 01:24:34.157508 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.157491 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" Jun 09 01:24:34.167919 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.167890 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5"] Jun 09 01:24:34.230558 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.230499 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx57d\" (UniqueName: \"kubernetes.io/projected/08b89027-6b0f-4a9e-9707-69990447e7c0-kube-api-access-zx57d\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5\" (UID: \"08b89027-6b0f-4a9e-9707-69990447e7c0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" Jun 09 01:24:34.230558 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.230561 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08b89027-6b0f-4a9e-9707-69990447e7c0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5\" (UID: \"08b89027-6b0f-4a9e-9707-69990447e7c0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" Jun 09 01:24:34.230763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.230663 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08b89027-6b0f-4a9e-9707-69990447e7c0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5\" (UID: \"08b89027-6b0f-4a9e-9707-69990447e7c0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" Jun 09 01:24:34.331894 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.331859 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08b89027-6b0f-4a9e-9707-69990447e7c0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5\" (UID: \"08b89027-6b0f-4a9e-9707-69990447e7c0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" Jun 09 01:24:34.332098 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.331916 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx57d\" (UniqueName: \"kubernetes.io/projected/08b89027-6b0f-4a9e-9707-69990447e7c0-kube-api-access-zx57d\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5\" (UID: \"08b89027-6b0f-4a9e-9707-69990447e7c0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" Jun 09 01:24:34.332098 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.331942 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08b89027-6b0f-4a9e-9707-69990447e7c0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5\" (UID: \"08b89027-6b0f-4a9e-9707-69990447e7c0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" Jun 09 01:24:34.332324 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.332301 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08b89027-6b0f-4a9e-9707-69990447e7c0-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5\" (UID: \"08b89027-6b0f-4a9e-9707-69990447e7c0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" Jun 09 01:24:34.332396 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.332310 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08b89027-6b0f-4a9e-9707-69990447e7c0-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5\" (UID: \"08b89027-6b0f-4a9e-9707-69990447e7c0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" Jun 09 01:24:34.341640 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.341610 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx57d\" (UniqueName: \"kubernetes.io/projected/08b89027-6b0f-4a9e-9707-69990447e7c0-kube-api-access-zx57d\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5\" (UID: \"08b89027-6b0f-4a9e-9707-69990447e7c0\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" Jun 09 01:24:34.468489 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.468401 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" Jun 09 01:24:34.551223 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.551197 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6"] Jun 09 01:24:34.554697 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.554674 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" Jun 09 01:24:34.562184 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.562162 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6"] Jun 09 01:24:34.604936 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.604905 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5"] Jun 09 01:24:34.605998 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:24:34.605971 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08b89027_6b0f_4a9e_9707_69990447e7c0.slice/crio-26972f359105e3de0aa3bde726e90b5f2f8554bd446b31d0c793e90bec10a714 WatchSource:0}: Error finding container 26972f359105e3de0aa3bde726e90b5f2f8554bd446b31d0c793e90bec10a714: Status 404 returned error can't find the container with id 26972f359105e3de0aa3bde726e90b5f2f8554bd446b31d0c793e90bec10a714 Jun 09 01:24:34.635476 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.635452 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6nzx\" (UniqueName: \"kubernetes.io/projected/5d3a828a-9d22-42a9-aa91-77290e064363-kube-api-access-x6nzx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6\" (UID: \"5d3a828a-9d22-42a9-aa91-77290e064363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" Jun 09 01:24:34.635610 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.635578 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d3a828a-9d22-42a9-aa91-77290e064363-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6\" (UID: \"5d3a828a-9d22-42a9-aa91-77290e064363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" Jun 09 01:24:34.635610 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.635599 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d3a828a-9d22-42a9-aa91-77290e064363-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6\" (UID: \"5d3a828a-9d22-42a9-aa91-77290e064363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" Jun 09 01:24:34.736405 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.736375 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d3a828a-9d22-42a9-aa91-77290e064363-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6\" (UID: \"5d3a828a-9d22-42a9-aa91-77290e064363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" Jun 09 01:24:34.736535 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.736413 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d3a828a-9d22-42a9-aa91-77290e064363-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6\" (UID: \"5d3a828a-9d22-42a9-aa91-77290e064363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" Jun 09 01:24:34.736535 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.736458 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6nzx\" (UniqueName: \"kubernetes.io/projected/5d3a828a-9d22-42a9-aa91-77290e064363-kube-api-access-x6nzx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6\" (UID: \"5d3a828a-9d22-42a9-aa91-77290e064363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" Jun 09 01:24:34.736784 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.736762 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d3a828a-9d22-42a9-aa91-77290e064363-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6\" (UID: \"5d3a828a-9d22-42a9-aa91-77290e064363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" Jun 09 01:24:34.736872 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.736849 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d3a828a-9d22-42a9-aa91-77290e064363-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6\" (UID: \"5d3a828a-9d22-42a9-aa91-77290e064363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" Jun 09 01:24:34.744452 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.744429 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6nzx\" (UniqueName: \"kubernetes.io/projected/5d3a828a-9d22-42a9-aa91-77290e064363-kube-api-access-x6nzx\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6\" (UID: \"5d3a828a-9d22-42a9-aa91-77290e064363\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" Jun 09 01:24:34.867470 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.867425 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" Jun 09 01:24:34.888213 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.888180 2555 generic.go:356] "Generic (PLEG): container finished" podID="2a52df31-fe8a-447f-93a8-d1409e9ab36e" containerID="c05c0bfc64f5ed0c38045002b9f2354145a40e06ee9857a5afcf4a1eff45caf5" exitCode=0 Jun 09 01:24:34.888403 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.888310 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" event={"ID":"2a52df31-fe8a-447f-93a8-d1409e9ab36e","Type":"ContainerDied","Data":"c05c0bfc64f5ed0c38045002b9f2354145a40e06ee9857a5afcf4a1eff45caf5"} Jun 09 01:24:34.890169 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.890142 2555 generic.go:356] "Generic (PLEG): container finished" podID="08b89027-6b0f-4a9e-9707-69990447e7c0" containerID="73cd7ae55ca0e0448b2690c1461d3f998c22694709102a66ac603f534133a44d" exitCode=0 Jun 09 01:24:34.890301 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.890180 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" event={"ID":"08b89027-6b0f-4a9e-9707-69990447e7c0","Type":"ContainerDied","Data":"73cd7ae55ca0e0448b2690c1461d3f998c22694709102a66ac603f534133a44d"} Jun 09 01:24:34.890301 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.890238 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" event={"ID":"08b89027-6b0f-4a9e-9707-69990447e7c0","Type":"ContainerStarted","Data":"26972f359105e3de0aa3bde726e90b5f2f8554bd446b31d0c793e90bec10a714"} Jun 09 01:24:34.956604 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.956571 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9"] Jun 09 01:24:34.960372 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.960352 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" Jun 09 01:24:34.968580 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:34.968535 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9"] Jun 09 01:24:35.008016 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.007991 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6"] Jun 09 01:24:35.009769 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:24:35.009739 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d3a828a_9d22_42a9_aa91_77290e064363.slice/crio-4fd6c4b068c5a3ceaceb7a83ea12712f8e3d72ec7ba80b84d446f6b9fb935d9f WatchSource:0}: Error finding container 4fd6c4b068c5a3ceaceb7a83ea12712f8e3d72ec7ba80b84d446f6b9fb935d9f: Status 404 returned error can't find the container with id 4fd6c4b068c5a3ceaceb7a83ea12712f8e3d72ec7ba80b84d446f6b9fb935d9f Jun 09 01:24:35.039597 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.039563 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfdvs\" (UniqueName: \"kubernetes.io/projected/4ae6be78-17e5-4020-9462-b3c3a292dec8-kube-api-access-dfdvs\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9\" (UID: \"4ae6be78-17e5-4020-9462-b3c3a292dec8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" Jun 09 01:24:35.039719 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.039620 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ae6be78-17e5-4020-9462-b3c3a292dec8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9\" (UID: \"4ae6be78-17e5-4020-9462-b3c3a292dec8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" Jun 09 01:24:35.039764 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.039716 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ae6be78-17e5-4020-9462-b3c3a292dec8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9\" (UID: \"4ae6be78-17e5-4020-9462-b3c3a292dec8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" Jun 09 01:24:35.141000 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.140970 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ae6be78-17e5-4020-9462-b3c3a292dec8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9\" (UID: \"4ae6be78-17e5-4020-9462-b3c3a292dec8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" Jun 09 01:24:35.141153 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.141040 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfdvs\" (UniqueName: \"kubernetes.io/projected/4ae6be78-17e5-4020-9462-b3c3a292dec8-kube-api-access-dfdvs\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9\" (UID: \"4ae6be78-17e5-4020-9462-b3c3a292dec8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" Jun 09 01:24:35.141153 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.141082 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ae6be78-17e5-4020-9462-b3c3a292dec8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9\" (UID: \"4ae6be78-17e5-4020-9462-b3c3a292dec8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" Jun 09 01:24:35.141335 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.141314 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ae6be78-17e5-4020-9462-b3c3a292dec8-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9\" (UID: \"4ae6be78-17e5-4020-9462-b3c3a292dec8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" Jun 09 01:24:35.141401 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.141340 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ae6be78-17e5-4020-9462-b3c3a292dec8-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9\" (UID: \"4ae6be78-17e5-4020-9462-b3c3a292dec8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" Jun 09 01:24:35.150273 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.150251 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfdvs\" (UniqueName: \"kubernetes.io/projected/4ae6be78-17e5-4020-9462-b3c3a292dec8-kube-api-access-dfdvs\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9\" (UID: \"4ae6be78-17e5-4020-9462-b3c3a292dec8\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" Jun 09 01:24:35.273666 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.273642 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" Jun 09 01:24:35.401818 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.401785 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9"] Jun 09 01:24:35.403316 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:24:35.403293 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ae6be78_17e5_4020_9462_b3c3a292dec8.slice/crio-e80889ea952a3c809520c3a42ef861d0410d77e0e9fc7a9a93e1360023182571 WatchSource:0}: Error finding container e80889ea952a3c809520c3a42ef861d0410d77e0e9fc7a9a93e1360023182571: Status 404 returned error can't find the container with id e80889ea952a3c809520c3a42ef861d0410d77e0e9fc7a9a93e1360023182571 Jun 09 01:24:35.899732 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.899702 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" event={"ID":"08b89027-6b0f-4a9e-9707-69990447e7c0","Type":"ContainerStarted","Data":"f0e067b2aaf7cecb5a2ebf2e132af1918931b1fb30ac5176f5cb17bdac400f24"} Jun 09 01:24:35.901486 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.901456 2555 generic.go:356] "Generic (PLEG): container finished" podID="4ae6be78-17e5-4020-9462-b3c3a292dec8" containerID="18e3ef2147c56b883c07679147d033f99b7d38f988abd59b3dd81a25b7883bf7" exitCode=0 Jun 09 01:24:35.901627 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.901521 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" event={"ID":"4ae6be78-17e5-4020-9462-b3c3a292dec8","Type":"ContainerDied","Data":"18e3ef2147c56b883c07679147d033f99b7d38f988abd59b3dd81a25b7883bf7"} Jun 09 01:24:35.901627 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.901586 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" event={"ID":"4ae6be78-17e5-4020-9462-b3c3a292dec8","Type":"ContainerStarted","Data":"e80889ea952a3c809520c3a42ef861d0410d77e0e9fc7a9a93e1360023182571"} Jun 09 01:24:35.903720 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.903493 2555 generic.go:356] "Generic (PLEG): container finished" podID="5d3a828a-9d22-42a9-aa91-77290e064363" containerID="0ab2f94e5b8ed5ef90b9140d2a563eca34d66f48bbb6f77ff009eb907f0b6b7a" exitCode=0 Jun 09 01:24:35.903720 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.903573 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" event={"ID":"5d3a828a-9d22-42a9-aa91-77290e064363","Type":"ContainerDied","Data":"0ab2f94e5b8ed5ef90b9140d2a563eca34d66f48bbb6f77ff009eb907f0b6b7a"} Jun 09 01:24:35.903720 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.903616 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" event={"ID":"5d3a828a-9d22-42a9-aa91-77290e064363","Type":"ContainerStarted","Data":"4fd6c4b068c5a3ceaceb7a83ea12712f8e3d72ec7ba80b84d446f6b9fb935d9f"} Jun 09 01:24:35.906795 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:35.906770 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" event={"ID":"2a52df31-fe8a-447f-93a8-d1409e9ab36e","Type":"ContainerStarted","Data":"bec585b7490dadc4495bd1d9e62870e7e81db462da19f2f686f98af937206ad7"} Jun 09 01:24:36.919986 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:36.919944 2555 generic.go:356] "Generic (PLEG): container finished" podID="2a52df31-fe8a-447f-93a8-d1409e9ab36e" containerID="bec585b7490dadc4495bd1d9e62870e7e81db462da19f2f686f98af937206ad7" exitCode=0 Jun 09 01:24:36.920339 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:36.920028 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" event={"ID":"2a52df31-fe8a-447f-93a8-d1409e9ab36e","Type":"ContainerDied","Data":"bec585b7490dadc4495bd1d9e62870e7e81db462da19f2f686f98af937206ad7"} Jun 09 01:24:36.921996 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:36.921971 2555 generic.go:356] "Generic (PLEG): container finished" podID="08b89027-6b0f-4a9e-9707-69990447e7c0" containerID="f0e067b2aaf7cecb5a2ebf2e132af1918931b1fb30ac5176f5cb17bdac400f24" exitCode=0 Jun 09 01:24:36.922099 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:36.922001 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" event={"ID":"08b89027-6b0f-4a9e-9707-69990447e7c0","Type":"ContainerDied","Data":"f0e067b2aaf7cecb5a2ebf2e132af1918931b1fb30ac5176f5cb17bdac400f24"} Jun 09 01:24:36.923909 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:36.923890 2555 generic.go:356] "Generic (PLEG): container finished" podID="4ae6be78-17e5-4020-9462-b3c3a292dec8" containerID="3d9cba391990697528a804f206f91ca438ed34b431890e546242be8387092d34" exitCode=0 Jun 09 01:24:36.924014 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:36.923966 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" event={"ID":"4ae6be78-17e5-4020-9462-b3c3a292dec8","Type":"ContainerDied","Data":"3d9cba391990697528a804f206f91ca438ed34b431890e546242be8387092d34"} Jun 09 01:24:36.925894 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:36.925875 2555 generic.go:356] "Generic (PLEG): container finished" podID="5d3a828a-9d22-42a9-aa91-77290e064363" containerID="4c6b068af167ddb8383cb221cde1fc50ab988a5199ed5ef051592319e1eafced" exitCode=0 Jun 09 01:24:36.925978 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:36.925960 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" event={"ID":"5d3a828a-9d22-42a9-aa91-77290e064363","Type":"ContainerDied","Data":"4c6b068af167ddb8383cb221cde1fc50ab988a5199ed5ef051592319e1eafced"} Jun 09 01:24:37.933932 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:37.933899 2555 generic.go:356] "Generic (PLEG): container finished" podID="2a52df31-fe8a-447f-93a8-d1409e9ab36e" containerID="e55b1acae8d5295ffd9ea96df6ed7f0f7a2f9b1e9b5c7e8b44738d94cf9e89a5" exitCode=0 Jun 09 01:24:37.934321 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:37.933984 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" event={"ID":"2a52df31-fe8a-447f-93a8-d1409e9ab36e","Type":"ContainerDied","Data":"e55b1acae8d5295ffd9ea96df6ed7f0f7a2f9b1e9b5c7e8b44738d94cf9e89a5"} Jun 09 01:24:37.935919 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:37.935897 2555 generic.go:356] "Generic (PLEG): container finished" podID="08b89027-6b0f-4a9e-9707-69990447e7c0" containerID="a519e88a32031b3cc050837dafb08ac9cc3433f48df137a1e3e4652cd6e0c5c8" exitCode=0 Jun 09 01:24:37.936041 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:37.935989 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" event={"ID":"08b89027-6b0f-4a9e-9707-69990447e7c0","Type":"ContainerDied","Data":"a519e88a32031b3cc050837dafb08ac9cc3433f48df137a1e3e4652cd6e0c5c8"} Jun 09 01:24:37.938066 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:37.938044 2555 generic.go:356] "Generic (PLEG): container finished" podID="4ae6be78-17e5-4020-9462-b3c3a292dec8" containerID="fb5250a9271d6aa8326c4beeb1c819a4bfd4452aaa91b275406057702af998b4" exitCode=0 Jun 09 01:24:37.938185 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:37.938102 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" event={"ID":"4ae6be78-17e5-4020-9462-b3c3a292dec8","Type":"ContainerDied","Data":"fb5250a9271d6aa8326c4beeb1c819a4bfd4452aaa91b275406057702af998b4"} Jun 09 01:24:37.940121 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:37.940102 2555 generic.go:356] "Generic (PLEG): container finished" podID="5d3a828a-9d22-42a9-aa91-77290e064363" containerID="c1541694e81476dd4feba2f5a30e4780ff4a23c23dfac53862ad50a504d1a2c6" exitCode=0 Jun 09 01:24:37.940230 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:37.940181 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" event={"ID":"5d3a828a-9d22-42a9-aa91-77290e064363","Type":"ContainerDied","Data":"c1541694e81476dd4feba2f5a30e4780ff4a23c23dfac53862ad50a504d1a2c6"} Jun 09 01:24:39.133319 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.133294 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" Jun 09 01:24:39.136829 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.136808 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" Jun 09 01:24:39.143379 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.143360 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" Jun 09 01:24:39.146311 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.146295 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" Jun 09 01:24:39.282748 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.282719 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a52df31-fe8a-447f-93a8-d1409e9ab36e-util\") pod \"2a52df31-fe8a-447f-93a8-d1409e9ab36e\" (UID: \"2a52df31-fe8a-447f-93a8-d1409e9ab36e\") " Jun 09 01:24:39.282923 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.282773 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24t9z\" (UniqueName: \"kubernetes.io/projected/2a52df31-fe8a-447f-93a8-d1409e9ab36e-kube-api-access-24t9z\") pod \"2a52df31-fe8a-447f-93a8-d1409e9ab36e\" (UID: \"2a52df31-fe8a-447f-93a8-d1409e9ab36e\") " Jun 09 01:24:39.282923 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.282802 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08b89027-6b0f-4a9e-9707-69990447e7c0-util\") pod \"08b89027-6b0f-4a9e-9707-69990447e7c0\" (UID: \"08b89027-6b0f-4a9e-9707-69990447e7c0\") " Jun 09 01:24:39.282923 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.282816 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d3a828a-9d22-42a9-aa91-77290e064363-util\") pod \"5d3a828a-9d22-42a9-aa91-77290e064363\" (UID: \"5d3a828a-9d22-42a9-aa91-77290e064363\") " Jun 09 01:24:39.282923 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.282829 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ae6be78-17e5-4020-9462-b3c3a292dec8-util\") pod \"4ae6be78-17e5-4020-9462-b3c3a292dec8\" (UID: \"4ae6be78-17e5-4020-9462-b3c3a292dec8\") " Jun 09 01:24:39.282923 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.282848 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a52df31-fe8a-447f-93a8-d1409e9ab36e-bundle\") pod \"2a52df31-fe8a-447f-93a8-d1409e9ab36e\" (UID: \"2a52df31-fe8a-447f-93a8-d1409e9ab36e\") " Jun 09 01:24:39.282923 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.282866 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08b89027-6b0f-4a9e-9707-69990447e7c0-bundle\") pod \"08b89027-6b0f-4a9e-9707-69990447e7c0\" (UID: \"08b89027-6b0f-4a9e-9707-69990447e7c0\") " Jun 09 01:24:39.282923 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.282893 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6nzx\" (UniqueName: \"kubernetes.io/projected/5d3a828a-9d22-42a9-aa91-77290e064363-kube-api-access-x6nzx\") pod \"5d3a828a-9d22-42a9-aa91-77290e064363\" (UID: \"5d3a828a-9d22-42a9-aa91-77290e064363\") " Jun 09 01:24:39.282923 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.282920 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfdvs\" (UniqueName: \"kubernetes.io/projected/4ae6be78-17e5-4020-9462-b3c3a292dec8-kube-api-access-dfdvs\") pod \"4ae6be78-17e5-4020-9462-b3c3a292dec8\" (UID: \"4ae6be78-17e5-4020-9462-b3c3a292dec8\") " Jun 09 01:24:39.283343 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.282982 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ae6be78-17e5-4020-9462-b3c3a292dec8-bundle\") pod \"4ae6be78-17e5-4020-9462-b3c3a292dec8\" (UID: \"4ae6be78-17e5-4020-9462-b3c3a292dec8\") " Jun 09 01:24:39.283343 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.283032 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zx57d\" (UniqueName: \"kubernetes.io/projected/08b89027-6b0f-4a9e-9707-69990447e7c0-kube-api-access-zx57d\") pod \"08b89027-6b0f-4a9e-9707-69990447e7c0\" (UID: \"08b89027-6b0f-4a9e-9707-69990447e7c0\") " Jun 09 01:24:39.283343 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.283063 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d3a828a-9d22-42a9-aa91-77290e064363-bundle\") pod \"5d3a828a-9d22-42a9-aa91-77290e064363\" (UID: \"5d3a828a-9d22-42a9-aa91-77290e064363\") " Jun 09 01:24:39.284138 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.284100 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a52df31-fe8a-447f-93a8-d1409e9ab36e-bundle" (OuterVolumeSpecName: "bundle") pod "2a52df31-fe8a-447f-93a8-d1409e9ab36e" (UID: "2a52df31-fe8a-447f-93a8-d1409e9ab36e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:24:39.284824 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.284525 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ae6be78-17e5-4020-9462-b3c3a292dec8-bundle" (OuterVolumeSpecName: "bundle") pod "4ae6be78-17e5-4020-9462-b3c3a292dec8" (UID: "4ae6be78-17e5-4020-9462-b3c3a292dec8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:24:39.285269 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.285249 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d3a828a-9d22-42a9-aa91-77290e064363-bundle" (OuterVolumeSpecName: "bundle") pod "5d3a828a-9d22-42a9-aa91-77290e064363" (UID: "5d3a828a-9d22-42a9-aa91-77290e064363"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:24:39.285973 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.285939 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ae6be78-17e5-4020-9462-b3c3a292dec8-kube-api-access-dfdvs" (OuterVolumeSpecName: "kube-api-access-dfdvs") pod "4ae6be78-17e5-4020-9462-b3c3a292dec8" (UID: "4ae6be78-17e5-4020-9462-b3c3a292dec8"). InnerVolumeSpecName "kube-api-access-dfdvs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:24:39.286072 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.285988 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a52df31-fe8a-447f-93a8-d1409e9ab36e-kube-api-access-24t9z" (OuterVolumeSpecName: "kube-api-access-24t9z") pod "2a52df31-fe8a-447f-93a8-d1409e9ab36e" (UID: "2a52df31-fe8a-447f-93a8-d1409e9ab36e"). InnerVolumeSpecName "kube-api-access-24t9z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:24:39.286248 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.286231 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08b89027-6b0f-4a9e-9707-69990447e7c0-bundle" (OuterVolumeSpecName: "bundle") pod "08b89027-6b0f-4a9e-9707-69990447e7c0" (UID: "08b89027-6b0f-4a9e-9707-69990447e7c0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:24:39.286699 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.286670 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08b89027-6b0f-4a9e-9707-69990447e7c0-kube-api-access-zx57d" (OuterVolumeSpecName: "kube-api-access-zx57d") pod "08b89027-6b0f-4a9e-9707-69990447e7c0" (UID: "08b89027-6b0f-4a9e-9707-69990447e7c0"). InnerVolumeSpecName "kube-api-access-zx57d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:24:39.287992 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.287967 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d3a828a-9d22-42a9-aa91-77290e064363-kube-api-access-x6nzx" (OuterVolumeSpecName: "kube-api-access-x6nzx") pod "5d3a828a-9d22-42a9-aa91-77290e064363" (UID: "5d3a828a-9d22-42a9-aa91-77290e064363"). InnerVolumeSpecName "kube-api-access-x6nzx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:24:39.289251 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.289230 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ae6be78-17e5-4020-9462-b3c3a292dec8-util" (OuterVolumeSpecName: "util") pod "4ae6be78-17e5-4020-9462-b3c3a292dec8" (UID: "4ae6be78-17e5-4020-9462-b3c3a292dec8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:24:39.289456 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.289434 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a52df31-fe8a-447f-93a8-d1409e9ab36e-util" (OuterVolumeSpecName: "util") pod "2a52df31-fe8a-447f-93a8-d1409e9ab36e" (UID: "2a52df31-fe8a-447f-93a8-d1409e9ab36e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:24:39.289840 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.289823 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08b89027-6b0f-4a9e-9707-69990447e7c0-util" (OuterVolumeSpecName: "util") pod "08b89027-6b0f-4a9e-9707-69990447e7c0" (UID: "08b89027-6b0f-4a9e-9707-69990447e7c0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:24:39.290255 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.290237 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d3a828a-9d22-42a9-aa91-77290e064363-util" (OuterVolumeSpecName: "util") pod "5d3a828a-9d22-42a9-aa91-77290e064363" (UID: "5d3a828a-9d22-42a9-aa91-77290e064363"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:24:39.383795 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.383769 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2a52df31-fe8a-447f-93a8-d1409e9ab36e-util\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:24:39.383795 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.383791 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24t9z\" (UniqueName: \"kubernetes.io/projected/2a52df31-fe8a-447f-93a8-d1409e9ab36e-kube-api-access-24t9z\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:24:39.383795 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.383801 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08b89027-6b0f-4a9e-9707-69990447e7c0-util\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:24:39.384020 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.383809 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5d3a828a-9d22-42a9-aa91-77290e064363-util\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:24:39.384020 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.383818 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ae6be78-17e5-4020-9462-b3c3a292dec8-util\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:24:39.384020 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.383825 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2a52df31-fe8a-447f-93a8-d1409e9ab36e-bundle\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:24:39.384020 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.383833 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08b89027-6b0f-4a9e-9707-69990447e7c0-bundle\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:24:39.384020 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.383841 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6nzx\" (UniqueName: \"kubernetes.io/projected/5d3a828a-9d22-42a9-aa91-77290e064363-kube-api-access-x6nzx\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:24:39.384020 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.383850 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfdvs\" (UniqueName: \"kubernetes.io/projected/4ae6be78-17e5-4020-9462-b3c3a292dec8-kube-api-access-dfdvs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:24:39.384020 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.383858 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ae6be78-17e5-4020-9462-b3c3a292dec8-bundle\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:24:39.384020 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.383867 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zx57d\" (UniqueName: \"kubernetes.io/projected/08b89027-6b0f-4a9e-9707-69990447e7c0-kube-api-access-zx57d\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:24:39.384020 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.383875 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5d3a828a-9d22-42a9-aa91-77290e064363-bundle\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:24:39.952772 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.952732 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" Jun 09 01:24:39.952772 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.952743 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2" event={"ID":"2a52df31-fe8a-447f-93a8-d1409e9ab36e","Type":"ContainerDied","Data":"973586ad34a43b9a2167eb4adc33c4dc0872952ab7a70ab7053d71a88d3e887a"} Jun 09 01:24:39.953007 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.952832 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="973586ad34a43b9a2167eb4adc33c4dc0872952ab7a70ab7053d71a88d3e887a" Jun 09 01:24:39.954792 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.954773 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" Jun 09 01:24:39.954792 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.954778 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5" event={"ID":"08b89027-6b0f-4a9e-9707-69990447e7c0","Type":"ContainerDied","Data":"26972f359105e3de0aa3bde726e90b5f2f8554bd446b31d0c793e90bec10a714"} Jun 09 01:24:39.954953 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.954820 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26972f359105e3de0aa3bde726e90b5f2f8554bd446b31d0c793e90bec10a714" Jun 09 01:24:39.957142 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.957126 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" Jun 09 01:24:39.957142 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.957126 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9" event={"ID":"4ae6be78-17e5-4020-9462-b3c3a292dec8","Type":"ContainerDied","Data":"e80889ea952a3c809520c3a42ef861d0410d77e0e9fc7a9a93e1360023182571"} Jun 09 01:24:39.957295 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.957160 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e80889ea952a3c809520c3a42ef861d0410d77e0e9fc7a9a93e1360023182571" Jun 09 01:24:39.959281 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.959266 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" Jun 09 01:24:39.959380 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.959283 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6" event={"ID":"5d3a828a-9d22-42a9-aa91-77290e064363","Type":"ContainerDied","Data":"4fd6c4b068c5a3ceaceb7a83ea12712f8e3d72ec7ba80b84d446f6b9fb935d9f"} Jun 09 01:24:39.959380 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:39.959312 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4fd6c4b068c5a3ceaceb7a83ea12712f8e3d72ec7ba80b84d446f6b9fb935d9f" Jun 09 01:24:46.565067 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565027 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85cd9cc44b-zl2mh"] Jun 09 01:24:46.565417 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565405 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08b89027-6b0f-4a9e-9707-69990447e7c0" containerName="pull" Jun 09 01:24:46.565417 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565416 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="08b89027-6b0f-4a9e-9707-69990447e7c0" containerName="pull" Jun 09 01:24:46.565496 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565429 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08b89027-6b0f-4a9e-9707-69990447e7c0" containerName="util" Jun 09 01:24:46.565496 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565435 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="08b89027-6b0f-4a9e-9707-69990447e7c0" containerName="util" Jun 09 01:24:46.565496 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565443 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d3a828a-9d22-42a9-aa91-77290e064363" containerName="util" Jun 09 01:24:46.565496 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565448 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3a828a-9d22-42a9-aa91-77290e064363" containerName="util" Jun 09 01:24:46.565496 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565454 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ae6be78-17e5-4020-9462-b3c3a292dec8" containerName="pull" Jun 09 01:24:46.565496 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565459 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ae6be78-17e5-4020-9462-b3c3a292dec8" containerName="pull" Jun 09 01:24:46.565496 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565470 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08b89027-6b0f-4a9e-9707-69990447e7c0" containerName="extract" Jun 09 01:24:46.565496 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565475 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="08b89027-6b0f-4a9e-9707-69990447e7c0" containerName="extract" Jun 09 01:24:46.565496 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565483 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a52df31-fe8a-447f-93a8-d1409e9ab36e" containerName="pull" Jun 09 01:24:46.565496 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565488 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a52df31-fe8a-447f-93a8-d1409e9ab36e" containerName="pull" Jun 09 01:24:46.565496 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565492 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d3a828a-9d22-42a9-aa91-77290e064363" containerName="pull" Jun 09 01:24:46.565496 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565497 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3a828a-9d22-42a9-aa91-77290e064363" containerName="pull" Jun 09 01:24:46.565873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565504 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a52df31-fe8a-447f-93a8-d1409e9ab36e" containerName="util" Jun 09 01:24:46.565873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565509 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a52df31-fe8a-447f-93a8-d1409e9ab36e" containerName="util" Jun 09 01:24:46.565873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565515 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d3a828a-9d22-42a9-aa91-77290e064363" containerName="extract" Jun 09 01:24:46.565873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565520 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3a828a-9d22-42a9-aa91-77290e064363" containerName="extract" Jun 09 01:24:46.565873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565524 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a52df31-fe8a-447f-93a8-d1409e9ab36e" containerName="extract" Jun 09 01:24:46.565873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565529 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a52df31-fe8a-447f-93a8-d1409e9ab36e" containerName="extract" Jun 09 01:24:46.565873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565555 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ae6be78-17e5-4020-9462-b3c3a292dec8" containerName="extract" Jun 09 01:24:46.565873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565563 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ae6be78-17e5-4020-9462-b3c3a292dec8" containerName="extract" Jun 09 01:24:46.565873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565585 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ae6be78-17e5-4020-9462-b3c3a292dec8" containerName="util" Jun 09 01:24:46.565873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565590 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ae6be78-17e5-4020-9462-b3c3a292dec8" containerName="util" Jun 09 01:24:46.565873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565650 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ae6be78-17e5-4020-9462-b3c3a292dec8" containerName="extract" Jun 09 01:24:46.565873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565659 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d3a828a-9d22-42a9-aa91-77290e064363" containerName="extract" Jun 09 01:24:46.565873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565666 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a52df31-fe8a-447f-93a8-d1409e9ab36e" containerName="extract" Jun 09 01:24:46.565873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.565674 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="08b89027-6b0f-4a9e-9707-69990447e7c0" containerName="extract" Jun 09 01:24:46.570530 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.570509 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.581082 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.581054 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85cd9cc44b-zl2mh"] Jun 09 01:24:46.653896 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.653861 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/129949c8-7227-45e4-99f7-618ee3d57270-console-config\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.653896 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.653898 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/129949c8-7227-45e4-99f7-618ee3d57270-service-ca\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.654089 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.653919 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/129949c8-7227-45e4-99f7-618ee3d57270-trusted-ca-bundle\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.654089 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.654007 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b27xg\" (UniqueName: \"kubernetes.io/projected/129949c8-7227-45e4-99f7-618ee3d57270-kube-api-access-b27xg\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.654089 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.654041 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/129949c8-7227-45e4-99f7-618ee3d57270-oauth-serving-cert\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.654089 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.654071 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/129949c8-7227-45e4-99f7-618ee3d57270-console-oauth-config\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.654260 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.654117 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/129949c8-7227-45e4-99f7-618ee3d57270-console-serving-cert\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.754948 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.754906 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/129949c8-7227-45e4-99f7-618ee3d57270-console-config\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.754948 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.754952 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/129949c8-7227-45e4-99f7-618ee3d57270-service-ca\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.755205 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.754973 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/129949c8-7227-45e4-99f7-618ee3d57270-trusted-ca-bundle\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.755205 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.755023 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b27xg\" (UniqueName: \"kubernetes.io/projected/129949c8-7227-45e4-99f7-618ee3d57270-kube-api-access-b27xg\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.755205 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.755052 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/129949c8-7227-45e4-99f7-618ee3d57270-oauth-serving-cert\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.755205 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.755087 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/129949c8-7227-45e4-99f7-618ee3d57270-console-oauth-config\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.755205 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.755139 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/129949c8-7227-45e4-99f7-618ee3d57270-console-serving-cert\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.755829 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.755802 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/129949c8-7227-45e4-99f7-618ee3d57270-console-config\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.755945 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.755802 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/129949c8-7227-45e4-99f7-618ee3d57270-oauth-serving-cert\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.755945 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.755854 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/129949c8-7227-45e4-99f7-618ee3d57270-service-ca\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.755945 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.755933 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/129949c8-7227-45e4-99f7-618ee3d57270-trusted-ca-bundle\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.757673 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.757642 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/129949c8-7227-45e4-99f7-618ee3d57270-console-oauth-config\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.757765 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.757648 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/129949c8-7227-45e4-99f7-618ee3d57270-console-serving-cert\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.763122 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.763101 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b27xg\" (UniqueName: \"kubernetes.io/projected/129949c8-7227-45e4-99f7-618ee3d57270-kube-api-access-b27xg\") pod \"console-85cd9cc44b-zl2mh\" (UID: \"129949c8-7227-45e4-99f7-618ee3d57270\") " pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:46.881326 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:46.881243 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:47.045569 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:47.045518 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85cd9cc44b-zl2mh"] Jun 09 01:24:47.046108 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:24:47.046076 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod129949c8_7227_45e4_99f7_618ee3d57270.slice/crio-3b211849541e23e3657f66c3d2dcfe400d39fc6180a1a1fced3d0179c68e2ec9 WatchSource:0}: Error finding container 3b211849541e23e3657f66c3d2dcfe400d39fc6180a1a1fced3d0179c68e2ec9: Status 404 returned error can't find the container with id 3b211849541e23e3657f66c3d2dcfe400d39fc6180a1a1fced3d0179c68e2ec9 Jun 09 01:24:48.010935 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:48.010896 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cd9cc44b-zl2mh" event={"ID":"129949c8-7227-45e4-99f7-618ee3d57270","Type":"ContainerStarted","Data":"6f31000174ab6c3b9b6d9fb03269db6282b2cf777d9d3764df4059c0804c2324"} Jun 09 01:24:48.011293 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:48.010957 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85cd9cc44b-zl2mh" event={"ID":"129949c8-7227-45e4-99f7-618ee3d57270","Type":"ContainerStarted","Data":"3b211849541e23e3657f66c3d2dcfe400d39fc6180a1a1fced3d0179c68e2ec9"} Jun 09 01:24:48.030571 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:48.030509 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85cd9cc44b-zl2mh" podStartSLOduration=2.030491831 podStartE2EDuration="2.030491831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:24:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:24:48.02982692 +0000 UTC m=+549.048131133" watchObservedRunningTime="2026-06-09 01:24:48.030491831 +0000 UTC m=+549.048796023" Jun 09 01:24:56.273366 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:56.273331 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2"] Jun 09 01:24:56.276944 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:56.276924 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" Jun 09 01:24:56.279953 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:56.279932 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 09 01:24:56.279953 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:56.279947 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 09 01:24:56.281175 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:56.281158 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-thj2s\"" Jun 09 01:24:56.291298 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:56.291267 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2"] Jun 09 01:24:56.340856 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:56.340825 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k29n\" (UniqueName: \"kubernetes.io/projected/75529a15-4886-40ca-9226-df412644feed-kube-api-access-7k29n\") pod \"limitador-operator-controller-manager-859d7b78bd-7j9r2\" (UID: \"75529a15-4886-40ca-9226-df412644feed\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" Jun 09 01:24:56.441308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:56.441278 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k29n\" (UniqueName: \"kubernetes.io/projected/75529a15-4886-40ca-9226-df412644feed-kube-api-access-7k29n\") pod \"limitador-operator-controller-manager-859d7b78bd-7j9r2\" (UID: \"75529a15-4886-40ca-9226-df412644feed\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" Jun 09 01:24:56.456719 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:56.456690 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k29n\" (UniqueName: \"kubernetes.io/projected/75529a15-4886-40ca-9226-df412644feed-kube-api-access-7k29n\") pod \"limitador-operator-controller-manager-859d7b78bd-7j9r2\" (UID: \"75529a15-4886-40ca-9226-df412644feed\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" Jun 09 01:24:56.592501 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:56.592440 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" Jun 09 01:24:56.729489 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:56.729464 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2"] Jun 09 01:24:56.732043 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:24:56.732005 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75529a15_4886_40ca_9226_df412644feed.slice/crio-647bb4c4bcf93875434c467fcfc3d7ecd502d6794e11e751c11b004278b1e1db WatchSource:0}: Error finding container 647bb4c4bcf93875434c467fcfc3d7ecd502d6794e11e751c11b004278b1e1db: Status 404 returned error can't find the container with id 647bb4c4bcf93875434c467fcfc3d7ecd502d6794e11e751c11b004278b1e1db Jun 09 01:24:56.881445 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:56.881383 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:56.881581 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:56.881446 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:56.886262 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:56.886241 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:57.064371 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:57.064342 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" event={"ID":"75529a15-4886-40ca-9226-df412644feed","Type":"ContainerStarted","Data":"647bb4c4bcf93875434c467fcfc3d7ecd502d6794e11e751c11b004278b1e1db"} Jun 09 01:24:57.067891 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:57.067871 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85cd9cc44b-zl2mh" Jun 09 01:24:57.120131 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:57.120103 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f9557d454-jtwcq"] Jun 09 01:24:59.078053 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:59.078018 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" event={"ID":"75529a15-4886-40ca-9226-df412644feed","Type":"ContainerStarted","Data":"c32a38aa0310768bcd79d70ed1b12be0554774c27b15e4c1187125e83267fcff"} Jun 09 01:24:59.078493 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:59.078137 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" Jun 09 01:24:59.097977 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:24:59.097933 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" podStartSLOduration=1.3636929580000001 podStartE2EDuration="3.097920821s" totalImagesPullingTime="1.734227863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:24:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:24:59.095086163 +0000 UTC m=+560.113390380" watchObservedRunningTime="2026-06-09 01:24:59.097920821 +0000 UTC m=+560.116225012" Jun 09 01:25:10.086660 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:10.086628 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" Jun 09 01:25:12.149569 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.149516 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2"] Jun 09 01:25:12.150029 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.149757 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" podUID="75529a15-4886-40ca-9226-df412644feed" containerName="manager" containerID="cri-o://c32a38aa0310768bcd79d70ed1b12be0554774c27b15e4c1187125e83267fcff" gracePeriod=2 Jun 09 01:25:12.165812 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.165777 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2"] Jun 09 01:25:12.176969 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:25:12.176929 2555 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"75529a15-4886-40ca-9226-df412644feed\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-7j9r2\": pods \"limitador-operator-controller-manager-859d7b78bd-7j9r2\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" Jun 09 01:25:12.179453 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.179431 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ddtm7"] Jun 09 01:25:12.179872 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.179858 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75529a15-4886-40ca-9226-df412644feed" containerName="manager" Jun 09 01:25:12.179918 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.179873 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="75529a15-4886-40ca-9226-df412644feed" containerName="manager" Jun 09 01:25:12.179963 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.179955 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="75529a15-4886-40ca-9226-df412644feed" containerName="manager" Jun 09 01:25:12.183142 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.183128 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ddtm7" Jun 09 01:25:12.185987 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:25:12.185950 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-7j9r2\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="75529a15-4886-40ca-9226-df412644feed" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" Jun 09 01:25:12.199037 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.199008 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ddtm7"] Jun 09 01:25:12.278066 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.278035 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l848\" (UniqueName: \"kubernetes.io/projected/bb07e658-88dc-45c8-9cda-7b195aec81da-kube-api-access-8l848\") pod \"limitador-operator-controller-manager-859d7b78bd-ddtm7\" (UID: \"bb07e658-88dc-45c8-9cda-7b195aec81da\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ddtm7" Jun 09 01:25:12.379421 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.379397 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l848\" (UniqueName: \"kubernetes.io/projected/bb07e658-88dc-45c8-9cda-7b195aec81da-kube-api-access-8l848\") pod \"limitador-operator-controller-manager-859d7b78bd-ddtm7\" (UID: \"bb07e658-88dc-45c8-9cda-7b195aec81da\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ddtm7" Jun 09 01:25:12.388362 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.388337 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l848\" (UniqueName: \"kubernetes.io/projected/bb07e658-88dc-45c8-9cda-7b195aec81da-kube-api-access-8l848\") pod \"limitador-operator-controller-manager-859d7b78bd-ddtm7\" (UID: \"bb07e658-88dc-45c8-9cda-7b195aec81da\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ddtm7" Jun 09 01:25:12.403601 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.403532 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" Jun 09 01:25:12.406061 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:25:12.406038 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-7j9r2\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="75529a15-4886-40ca-9226-df412644feed" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" Jun 09 01:25:12.480070 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.480040 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7k29n\" (UniqueName: \"kubernetes.io/projected/75529a15-4886-40ca-9226-df412644feed-kube-api-access-7k29n\") pod \"75529a15-4886-40ca-9226-df412644feed\" (UID: \"75529a15-4886-40ca-9226-df412644feed\") " Jun 09 01:25:12.482017 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.481997 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75529a15-4886-40ca-9226-df412644feed-kube-api-access-7k29n" (OuterVolumeSpecName: "kube-api-access-7k29n") pod "75529a15-4886-40ca-9226-df412644feed" (UID: "75529a15-4886-40ca-9226-df412644feed"). InnerVolumeSpecName "kube-api-access-7k29n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:25:12.548364 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.548327 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ddtm7" Jun 09 01:25:12.581131 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.581092 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7k29n\" (UniqueName: \"kubernetes.io/projected/75529a15-4886-40ca-9226-df412644feed-kube-api-access-7k29n\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:25:12.689227 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:12.689201 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ddtm7"] Jun 09 01:25:12.689694 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:25:12.689661 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb07e658_88dc_45c8_9cda_7b195aec81da.slice/crio-9ce432fcc79f712b397b0153260bd3d2b3ff20ae843ab8d31454868222298a94 WatchSource:0}: Error finding container 9ce432fcc79f712b397b0153260bd3d2b3ff20ae843ab8d31454868222298a94: Status 404 returned error can't find the container with id 9ce432fcc79f712b397b0153260bd3d2b3ff20ae843ab8d31454868222298a94 Jun 09 01:25:13.172720 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:13.172672 2555 generic.go:356] "Generic (PLEG): container finished" podID="75529a15-4886-40ca-9226-df412644feed" containerID="c32a38aa0310768bcd79d70ed1b12be0554774c27b15e4c1187125e83267fcff" exitCode=0 Jun 09 01:25:13.173126 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:13.172733 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" Jun 09 01:25:13.173126 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:13.172758 2555 scope.go:117] "RemoveContainer" containerID="c32a38aa0310768bcd79d70ed1b12be0554774c27b15e4c1187125e83267fcff" Jun 09 01:25:13.174705 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:13.174684 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ddtm7" event={"ID":"bb07e658-88dc-45c8-9cda-7b195aec81da","Type":"ContainerStarted","Data":"c556ab99e719fe185499965e0bb55fda9b1231bde2edabe688d410323eb698ab"} Jun 09 01:25:13.174810 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:13.174750 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ddtm7" event={"ID":"bb07e658-88dc-45c8-9cda-7b195aec81da","Type":"ContainerStarted","Data":"9ce432fcc79f712b397b0153260bd3d2b3ff20ae843ab8d31454868222298a94"} Jun 09 01:25:13.174810 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:13.174772 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ddtm7" Jun 09 01:25:13.179760 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:25:13.179733 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-7j9r2\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="75529a15-4886-40ca-9226-df412644feed" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" Jun 09 01:25:13.182319 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:25:13.182296 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-7j9r2\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="75529a15-4886-40ca-9226-df412644feed" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" Jun 09 01:25:13.183406 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:13.183389 2555 scope.go:117] "RemoveContainer" containerID="c32a38aa0310768bcd79d70ed1b12be0554774c27b15e4c1187125e83267fcff" Jun 09 01:25:13.183670 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:25:13.183651 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c32a38aa0310768bcd79d70ed1b12be0554774c27b15e4c1187125e83267fcff\": container with ID starting with c32a38aa0310768bcd79d70ed1b12be0554774c27b15e4c1187125e83267fcff not found: ID does not exist" containerID="c32a38aa0310768bcd79d70ed1b12be0554774c27b15e4c1187125e83267fcff" Jun 09 01:25:13.183738 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:13.183678 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c32a38aa0310768bcd79d70ed1b12be0554774c27b15e4c1187125e83267fcff"} err="failed to get container status \"c32a38aa0310768bcd79d70ed1b12be0554774c27b15e4c1187125e83267fcff\": rpc error: code = NotFound desc = could not find container \"c32a38aa0310768bcd79d70ed1b12be0554774c27b15e4c1187125e83267fcff\": container with ID starting with c32a38aa0310768bcd79d70ed1b12be0554774c27b15e4c1187125e83267fcff not found: ID does not exist" Jun 09 01:25:13.203759 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:25:13.203730 2555 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-7j9r2\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="75529a15-4886-40ca-9226-df412644feed" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7j9r2" Jun 09 01:25:13.204290 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:13.204252 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ddtm7" podStartSLOduration=1.204239324 podStartE2EDuration="1.204239324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:25:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:25:13.201055877 +0000 UTC m=+574.219360067" watchObservedRunningTime="2026-06-09 01:25:13.204239324 +0000 UTC m=+574.222543516" Jun 09 01:25:13.416877 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:13.416842 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75529a15-4886-40ca-9226-df412644feed" path="/var/lib/kubelet/pods/75529a15-4886-40ca-9226-df412644feed/volumes" Jun 09 01:25:22.140288 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.140240 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5f9557d454-jtwcq" podUID="8b554804-73a6-4a4a-b948-46dabe1ade54" containerName="console" containerID="cri-o://cab1ca7a4a2a5729921e03e36f4bb9362652a6277b46b303427cb16dc9a4cfd1" gracePeriod=15 Jun 09 01:25:22.390657 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.390595 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:25:22.476650 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.476619 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbxps\" (UniqueName: \"kubernetes.io/projected/8b554804-73a6-4a4a-b948-46dabe1ade54-kube-api-access-vbxps\") pod \"8b554804-73a6-4a4a-b948-46dabe1ade54\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " Jun 09 01:25:22.476850 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.476737 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b554804-73a6-4a4a-b948-46dabe1ade54-console-serving-cert\") pod \"8b554804-73a6-4a4a-b948-46dabe1ade54\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " Jun 09 01:25:22.476850 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.476773 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b554804-73a6-4a4a-b948-46dabe1ade54-console-oauth-config\") pod \"8b554804-73a6-4a4a-b948-46dabe1ade54\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " Jun 09 01:25:22.476850 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.476808 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-console-config\") pod \"8b554804-73a6-4a4a-b948-46dabe1ade54\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " Jun 09 01:25:22.476850 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.476837 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-oauth-serving-cert\") pod \"8b554804-73a6-4a4a-b948-46dabe1ade54\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " Jun 09 01:25:22.477071 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.476864 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-service-ca\") pod \"8b554804-73a6-4a4a-b948-46dabe1ade54\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " Jun 09 01:25:22.477071 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.476930 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-trusted-ca-bundle\") pod \"8b554804-73a6-4a4a-b948-46dabe1ade54\" (UID: \"8b554804-73a6-4a4a-b948-46dabe1ade54\") " Jun 09 01:25:22.477286 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.477248 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8b554804-73a6-4a4a-b948-46dabe1ade54" (UID: "8b554804-73a6-4a4a-b948-46dabe1ade54"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 01:25:22.477286 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.477266 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-console-config" (OuterVolumeSpecName: "console-config") pod "8b554804-73a6-4a4a-b948-46dabe1ade54" (UID: "8b554804-73a6-4a4a-b948-46dabe1ade54"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 01:25:22.477481 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.477447 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-service-ca" (OuterVolumeSpecName: "service-ca") pod "8b554804-73a6-4a4a-b948-46dabe1ade54" (UID: "8b554804-73a6-4a4a-b948-46dabe1ade54"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 01:25:22.477639 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.477515 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8b554804-73a6-4a4a-b948-46dabe1ade54" (UID: "8b554804-73a6-4a4a-b948-46dabe1ade54"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 01:25:22.478941 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.478913 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b554804-73a6-4a4a-b948-46dabe1ade54-kube-api-access-vbxps" (OuterVolumeSpecName: "kube-api-access-vbxps") pod "8b554804-73a6-4a4a-b948-46dabe1ade54" (UID: "8b554804-73a6-4a4a-b948-46dabe1ade54"). InnerVolumeSpecName "kube-api-access-vbxps". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:25:22.479027 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.478948 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b554804-73a6-4a4a-b948-46dabe1ade54-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8b554804-73a6-4a4a-b948-46dabe1ade54" (UID: "8b554804-73a6-4a4a-b948-46dabe1ade54"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 01:25:22.479027 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.478982 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b554804-73a6-4a4a-b948-46dabe1ade54-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8b554804-73a6-4a4a-b948-46dabe1ade54" (UID: "8b554804-73a6-4a4a-b948-46dabe1ade54"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 01:25:22.578524 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.578491 2555 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8b554804-73a6-4a4a-b948-46dabe1ade54-console-serving-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:25:22.578524 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.578521 2555 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8b554804-73a6-4a4a-b948-46dabe1ade54-console-oauth-config\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:25:22.578752 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.578534 2555 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-console-config\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:25:22.578752 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.578570 2555 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-oauth-serving-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:25:22.578752 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.578579 2555 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-service-ca\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:25:22.578752 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.578587 2555 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b554804-73a6-4a4a-b948-46dabe1ade54-trusted-ca-bundle\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:25:22.578752 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:22.578596 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbxps\" (UniqueName: \"kubernetes.io/projected/8b554804-73a6-4a4a-b948-46dabe1ade54-kube-api-access-vbxps\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:25:23.235798 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:23.235758 2555 generic.go:356] "Generic (PLEG): container finished" podID="8b554804-73a6-4a4a-b948-46dabe1ade54" containerID="cab1ca7a4a2a5729921e03e36f4bb9362652a6277b46b303427cb16dc9a4cfd1" exitCode=0 Jun 09 01:25:23.235798 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:23.235804 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f9557d454-jtwcq" Jun 09 01:25:23.236227 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:23.235812 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f9557d454-jtwcq" event={"ID":"8b554804-73a6-4a4a-b948-46dabe1ade54","Type":"ContainerDied","Data":"cab1ca7a4a2a5729921e03e36f4bb9362652a6277b46b303427cb16dc9a4cfd1"} Jun 09 01:25:23.236227 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:23.235864 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f9557d454-jtwcq" event={"ID":"8b554804-73a6-4a4a-b948-46dabe1ade54","Type":"ContainerDied","Data":"7c956fd07047cc71a8cf48698b1f197f64bd78944616e25d41cbe2abfd835e23"} Jun 09 01:25:23.236227 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:23.235890 2555 scope.go:117] "RemoveContainer" containerID="cab1ca7a4a2a5729921e03e36f4bb9362652a6277b46b303427cb16dc9a4cfd1" Jun 09 01:25:23.245867 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:23.245848 2555 scope.go:117] "RemoveContainer" containerID="cab1ca7a4a2a5729921e03e36f4bb9362652a6277b46b303427cb16dc9a4cfd1" Jun 09 01:25:23.246122 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:25:23.246105 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cab1ca7a4a2a5729921e03e36f4bb9362652a6277b46b303427cb16dc9a4cfd1\": container with ID starting with cab1ca7a4a2a5729921e03e36f4bb9362652a6277b46b303427cb16dc9a4cfd1 not found: ID does not exist" containerID="cab1ca7a4a2a5729921e03e36f4bb9362652a6277b46b303427cb16dc9a4cfd1" Jun 09 01:25:23.246180 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:23.246129 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cab1ca7a4a2a5729921e03e36f4bb9362652a6277b46b303427cb16dc9a4cfd1"} err="failed to get container status \"cab1ca7a4a2a5729921e03e36f4bb9362652a6277b46b303427cb16dc9a4cfd1\": rpc error: code = NotFound desc = could not find container \"cab1ca7a4a2a5729921e03e36f4bb9362652a6277b46b303427cb16dc9a4cfd1\": container with ID starting with cab1ca7a4a2a5729921e03e36f4bb9362652a6277b46b303427cb16dc9a4cfd1 not found: ID does not exist" Jun 09 01:25:23.258745 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:23.258723 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f9557d454-jtwcq"] Jun 09 01:25:23.262974 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:23.262950 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f9557d454-jtwcq"] Jun 09 01:25:23.416285 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:23.416251 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b554804-73a6-4a4a-b948-46dabe1ade54" path="/var/lib/kubelet/pods/8b554804-73a6-4a4a-b948-46dabe1ade54/volumes" Jun 09 01:25:24.184616 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:24.184586 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ddtm7" Jun 09 01:25:39.410145 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:39.410013 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:25:39.432916 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:39.411607 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:25:39.432916 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:39.415565 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:25:39.432916 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:39.417190 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:25:54.008282 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:54.008187 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-blrqg"] Jun 09 01:25:54.008702 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:54.008578 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b554804-73a6-4a4a-b948-46dabe1ade54" containerName="console" Jun 09 01:25:54.008702 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:54.008590 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b554804-73a6-4a4a-b948-46dabe1ade54" containerName="console" Jun 09 01:25:54.008702 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:54.008680 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b554804-73a6-4a4a-b948-46dabe1ade54" containerName="console" Jun 09 01:25:54.011626 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:54.011609 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" Jun 09 01:25:54.014476 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:54.014453 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 09 01:25:54.014618 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:54.014459 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-f84hw\"" Jun 09 01:25:54.020084 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:54.019501 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-blrqg"] Jun 09 01:25:54.105599 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:54.105536 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-blrqg"] Jun 09 01:25:54.146963 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:54.146926 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8knb8\" (UniqueName: \"kubernetes.io/projected/84942097-ee0a-44bd-ad79-15b57325ce32-kube-api-access-8knb8\") pod \"limitador-limitador-64c94fd64f-blrqg\" (UID: \"84942097-ee0a-44bd-ad79-15b57325ce32\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" Jun 09 01:25:54.147158 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:54.146971 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/84942097-ee0a-44bd-ad79-15b57325ce32-config-file\") pod \"limitador-limitador-64c94fd64f-blrqg\" (UID: \"84942097-ee0a-44bd-ad79-15b57325ce32\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" Jun 09 01:25:54.248043 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:54.247999 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8knb8\" (UniqueName: \"kubernetes.io/projected/84942097-ee0a-44bd-ad79-15b57325ce32-kube-api-access-8knb8\") pod \"limitador-limitador-64c94fd64f-blrqg\" (UID: \"84942097-ee0a-44bd-ad79-15b57325ce32\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" Jun 09 01:25:54.248216 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:54.248067 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/84942097-ee0a-44bd-ad79-15b57325ce32-config-file\") pod \"limitador-limitador-64c94fd64f-blrqg\" (UID: \"84942097-ee0a-44bd-ad79-15b57325ce32\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" Jun 09 01:25:54.248775 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:54.248746 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/84942097-ee0a-44bd-ad79-15b57325ce32-config-file\") pod \"limitador-limitador-64c94fd64f-blrqg\" (UID: \"84942097-ee0a-44bd-ad79-15b57325ce32\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" Jun 09 01:25:54.256835 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:54.256809 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8knb8\" (UniqueName: \"kubernetes.io/projected/84942097-ee0a-44bd-ad79-15b57325ce32-kube-api-access-8knb8\") pod \"limitador-limitador-64c94fd64f-blrqg\" (UID: \"84942097-ee0a-44bd-ad79-15b57325ce32\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" Jun 09 01:25:54.323749 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:54.323710 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" Jun 09 01:25:54.468155 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:54.468121 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-blrqg"] Jun 09 01:25:55.436518 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:55.436432 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" event={"ID":"84942097-ee0a-44bd-ad79-15b57325ce32","Type":"ContainerStarted","Data":"883d941bfd706b4d06debed831324d22b1fd4d8e0955acef6c90c078c8b5a3c8"} Jun 09 01:25:57.452595 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:57.452557 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" event={"ID":"84942097-ee0a-44bd-ad79-15b57325ce32","Type":"ContainerStarted","Data":"ff37611dd748affbca9a44a399893c1021415c2f02c7acdab5962ae9fbbab3be"} Jun 09 01:25:57.453040 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:57.452704 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" Jun 09 01:25:57.470881 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:25:57.470773 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" podStartSLOduration=1.7342004260000001 podStartE2EDuration="4.470758245s" totalImagesPullingTime="2.736557819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:25:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:25:57.467753282 +0000 UTC m=+618.486057503" watchObservedRunningTime="2026-06-09 01:25:57.470758245 +0000 UTC m=+618.489062437" Jun 09 01:26:08.459782 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:08.459753 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" Jun 09 01:26:08.860656 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:08.860619 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-blrqg"] Jun 09 01:26:08.860863 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:08.860840 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" podUID="84942097-ee0a-44bd-ad79-15b57325ce32" containerName="limitador" containerID="cri-o://ff37611dd748affbca9a44a399893c1021415c2f02c7acdab5962ae9fbbab3be" gracePeriod=30 Jun 09 01:26:09.523516 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:09.523486 2555 generic.go:356] "Generic (PLEG): container finished" podID="84942097-ee0a-44bd-ad79-15b57325ce32" containerID="ff37611dd748affbca9a44a399893c1021415c2f02c7acdab5962ae9fbbab3be" exitCode=0 Jun 09 01:26:09.523912 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:09.523577 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" event={"ID":"84942097-ee0a-44bd-ad79-15b57325ce32","Type":"ContainerDied","Data":"ff37611dd748affbca9a44a399893c1021415c2f02c7acdab5962ae9fbbab3be"} Jun 09 01:26:09.823466 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:09.823442 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" Jun 09 01:26:09.993298 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:09.993260 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8knb8\" (UniqueName: \"kubernetes.io/projected/84942097-ee0a-44bd-ad79-15b57325ce32-kube-api-access-8knb8\") pod \"84942097-ee0a-44bd-ad79-15b57325ce32\" (UID: \"84942097-ee0a-44bd-ad79-15b57325ce32\") " Jun 09 01:26:09.993504 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:09.993379 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/84942097-ee0a-44bd-ad79-15b57325ce32-config-file\") pod \"84942097-ee0a-44bd-ad79-15b57325ce32\" (UID: \"84942097-ee0a-44bd-ad79-15b57325ce32\") " Jun 09 01:26:09.993784 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:09.993759 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84942097-ee0a-44bd-ad79-15b57325ce32-config-file" (OuterVolumeSpecName: "config-file") pod "84942097-ee0a-44bd-ad79-15b57325ce32" (UID: "84942097-ee0a-44bd-ad79-15b57325ce32"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 01:26:09.995419 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:09.995394 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84942097-ee0a-44bd-ad79-15b57325ce32-kube-api-access-8knb8" (OuterVolumeSpecName: "kube-api-access-8knb8") pod "84942097-ee0a-44bd-ad79-15b57325ce32" (UID: "84942097-ee0a-44bd-ad79-15b57325ce32"). InnerVolumeSpecName "kube-api-access-8knb8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:26:10.095130 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:10.095030 2555 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/84942097-ee0a-44bd-ad79-15b57325ce32-config-file\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:26:10.095130 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:10.095075 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8knb8\" (UniqueName: \"kubernetes.io/projected/84942097-ee0a-44bd-ad79-15b57325ce32-kube-api-access-8knb8\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:26:10.530837 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:10.530813 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" Jun 09 01:26:10.531256 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:10.530810 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-blrqg" event={"ID":"84942097-ee0a-44bd-ad79-15b57325ce32","Type":"ContainerDied","Data":"883d941bfd706b4d06debed831324d22b1fd4d8e0955acef6c90c078c8b5a3c8"} Jun 09 01:26:10.531256 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:10.530954 2555 scope.go:117] "RemoveContainer" containerID="ff37611dd748affbca9a44a399893c1021415c2f02c7acdab5962ae9fbbab3be" Jun 09 01:26:10.553187 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:10.553163 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-blrqg"] Jun 09 01:26:10.557113 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:10.557091 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-blrqg"] Jun 09 01:26:11.416728 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.416696 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84942097-ee0a-44bd-ad79-15b57325ce32" path="/var/lib/kubelet/pods/84942097-ee0a-44bd-ad79-15b57325ce32/volumes" Jun 09 01:26:11.570886 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.570845 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["redhat-ai-gateway-infra/postgres-69fb4594cf-4mzgc"] Jun 09 01:26:11.571519 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.571500 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84942097-ee0a-44bd-ad79-15b57325ce32" containerName="limitador" Jun 09 01:26:11.571598 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.571524 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="84942097-ee0a-44bd-ad79-15b57325ce32" containerName="limitador" Jun 09 01:26:11.571673 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.571659 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="84942097-ee0a-44bd-ad79-15b57325ce32" containerName="limitador" Jun 09 01:26:11.576839 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.576651 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-4mzgc" Jun 09 01:26:11.581340 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.581312 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/postgres-69fb4594cf-4mzgc"] Jun 09 01:26:11.581919 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.581710 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jun 09 01:26:11.581919 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.581763 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"redhat-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jun 09 01:26:11.582098 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.582039 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"postgres-creds\"" Jun 09 01:26:11.582261 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.582237 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"default-dockercfg-d9cqp\"" Jun 09 01:26:11.708224 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.708134 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a783b42a-ece2-4904-87be-9403ae6c1e64-data\") pod \"postgres-69fb4594cf-4mzgc\" (UID: \"a783b42a-ece2-4904-87be-9403ae6c1e64\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-4mzgc" Jun 09 01:26:11.708224 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.708180 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp8bm\" (UniqueName: \"kubernetes.io/projected/a783b42a-ece2-4904-87be-9403ae6c1e64-kube-api-access-bp8bm\") pod \"postgres-69fb4594cf-4mzgc\" (UID: \"a783b42a-ece2-4904-87be-9403ae6c1e64\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-4mzgc" Jun 09 01:26:11.808817 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.808776 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a783b42a-ece2-4904-87be-9403ae6c1e64-data\") pod \"postgres-69fb4594cf-4mzgc\" (UID: \"a783b42a-ece2-4904-87be-9403ae6c1e64\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-4mzgc" Jun 09 01:26:11.808817 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.808822 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp8bm\" (UniqueName: \"kubernetes.io/projected/a783b42a-ece2-4904-87be-9403ae6c1e64-kube-api-access-bp8bm\") pod \"postgres-69fb4594cf-4mzgc\" (UID: \"a783b42a-ece2-4904-87be-9403ae6c1e64\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-4mzgc" Jun 09 01:26:11.809176 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.809154 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a783b42a-ece2-4904-87be-9403ae6c1e64-data\") pod \"postgres-69fb4594cf-4mzgc\" (UID: \"a783b42a-ece2-4904-87be-9403ae6c1e64\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-4mzgc" Jun 09 01:26:11.816997 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.816974 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp8bm\" (UniqueName: \"kubernetes.io/projected/a783b42a-ece2-4904-87be-9403ae6c1e64-kube-api-access-bp8bm\") pod \"postgres-69fb4594cf-4mzgc\" (UID: \"a783b42a-ece2-4904-87be-9403ae6c1e64\") " pod="redhat-ai-gateway-infra/postgres-69fb4594cf-4mzgc" Jun 09 01:26:11.889897 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:11.889863 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-4mzgc" Jun 09 01:26:12.031018 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:12.030989 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/postgres-69fb4594cf-4mzgc"] Jun 09 01:26:12.031739 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:26:12.031717 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda783b42a_ece2_4904_87be_9403ae6c1e64.slice/crio-553bef3e8c0092141f839ca488d73ce60884205d5157d6c7e0ac6fa6403d9d0a WatchSource:0}: Error finding container 553bef3e8c0092141f839ca488d73ce60884205d5157d6c7e0ac6fa6403d9d0a: Status 404 returned error can't find the container with id 553bef3e8c0092141f839ca488d73ce60884205d5157d6c7e0ac6fa6403d9d0a Jun 09 01:26:12.544185 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:12.544142 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-4mzgc" event={"ID":"a783b42a-ece2-4904-87be-9403ae6c1e64","Type":"ContainerStarted","Data":"553bef3e8c0092141f839ca488d73ce60884205d5157d6c7e0ac6fa6403d9d0a"} Jun 09 01:26:14.559173 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:14.559128 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-4mzgc" event={"ID":"a783b42a-ece2-4904-87be-9403ae6c1e64","Type":"ContainerStarted","Data":"2ec9e635da0f257506c4546df75670e1be5661e4fc7534e303a5ff30db44cac4"} Jun 09 01:26:14.559601 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:14.559298 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-4mzgc" Jun 09 01:26:14.577367 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:14.577316 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-4mzgc" podStartSLOduration=1.42970958 podStartE2EDuration="3.577301521s" totalImagesPullingTime="2.147591941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:26:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:26:14.575790807 +0000 UTC m=+635.594094999" watchObservedRunningTime="2026-06-09 01:26:14.577301521 +0000 UTC m=+635.595605712" Jun 09 01:26:20.600801 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:20.600775 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ai-gateway-infra/postgres-69fb4594cf-4mzgc" Jun 09 01:26:21.948988 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:21.948948 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb"] Jun 09 01:26:21.955715 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:21.955687 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" Jun 09 01:26:21.961374 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:21.961335 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 09 01:26:21.962448 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:21.962419 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 09 01:26:21.964971 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:21.964942 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6rf6w\"" Jun 09 01:26:21.965916 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:21.965889 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb"] Jun 09 01:26:22.000287 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:22.000249 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ebf50a7-9ab0-4775-9a34-874f5518b565-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb\" (UID: \"5ebf50a7-9ab0-4775-9a34-874f5518b565\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" Jun 09 01:26:22.000437 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:22.000307 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ebf50a7-9ab0-4775-9a34-874f5518b565-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb\" (UID: \"5ebf50a7-9ab0-4775-9a34-874f5518b565\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" Jun 09 01:26:22.000437 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:22.000358 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2jr2\" (UniqueName: \"kubernetes.io/projected/5ebf50a7-9ab0-4775-9a34-874f5518b565-kube-api-access-h2jr2\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb\" (UID: \"5ebf50a7-9ab0-4775-9a34-874f5518b565\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" Jun 09 01:26:22.100915 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:22.100878 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ebf50a7-9ab0-4775-9a34-874f5518b565-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb\" (UID: \"5ebf50a7-9ab0-4775-9a34-874f5518b565\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" Jun 09 01:26:22.101078 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:22.100943 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ebf50a7-9ab0-4775-9a34-874f5518b565-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb\" (UID: \"5ebf50a7-9ab0-4775-9a34-874f5518b565\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" Jun 09 01:26:22.101078 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:22.100974 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2jr2\" (UniqueName: \"kubernetes.io/projected/5ebf50a7-9ab0-4775-9a34-874f5518b565-kube-api-access-h2jr2\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb\" (UID: \"5ebf50a7-9ab0-4775-9a34-874f5518b565\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" Jun 09 01:26:22.101245 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:22.101226 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ebf50a7-9ab0-4775-9a34-874f5518b565-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb\" (UID: \"5ebf50a7-9ab0-4775-9a34-874f5518b565\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" Jun 09 01:26:22.101282 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:22.101252 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ebf50a7-9ab0-4775-9a34-874f5518b565-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb\" (UID: \"5ebf50a7-9ab0-4775-9a34-874f5518b565\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" Jun 09 01:26:22.115479 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:22.115457 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2jr2\" (UniqueName: \"kubernetes.io/projected/5ebf50a7-9ab0-4775-9a34-874f5518b565-kube-api-access-h2jr2\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb\" (UID: \"5ebf50a7-9ab0-4775-9a34-874f5518b565\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" Jun 09 01:26:22.273456 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:22.273429 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" Jun 09 01:26:22.412525 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:22.412498 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb"] Jun 09 01:26:22.413371 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:26:22.413344 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ebf50a7_9ab0_4775_9a34_874f5518b565.slice/crio-c5cf03895da3c308eebab8ecf6d074f9e4c471e626432af490e1eb9147a4fa25 WatchSource:0}: Error finding container c5cf03895da3c308eebab8ecf6d074f9e4c471e626432af490e1eb9147a4fa25: Status 404 returned error can't find the container with id c5cf03895da3c308eebab8ecf6d074f9e4c471e626432af490e1eb9147a4fa25 Jun 09 01:26:22.605203 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:22.605166 2555 generic.go:356] "Generic (PLEG): container finished" podID="5ebf50a7-9ab0-4775-9a34-874f5518b565" containerID="bfa5089a778a1729a1c3e973214821e80e12d88f3814e71ebe8e921a87a2b60d" exitCode=0 Jun 09 01:26:22.605359 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:22.605248 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" event={"ID":"5ebf50a7-9ab0-4775-9a34-874f5518b565","Type":"ContainerDied","Data":"bfa5089a778a1729a1c3e973214821e80e12d88f3814e71ebe8e921a87a2b60d"} Jun 09 01:26:22.605359 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:22.605297 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" event={"ID":"5ebf50a7-9ab0-4775-9a34-874f5518b565","Type":"ContainerStarted","Data":"c5cf03895da3c308eebab8ecf6d074f9e4c471e626432af490e1eb9147a4fa25"} Jun 09 01:26:23.612643 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:23.612606 2555 generic.go:356] "Generic (PLEG): container finished" podID="5ebf50a7-9ab0-4775-9a34-874f5518b565" containerID="b7ae2cf456a72320ee2d2adface7ec28e1a85c0942a156c384a0b841bb07af8a" exitCode=0 Jun 09 01:26:23.613037 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:23.612666 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" event={"ID":"5ebf50a7-9ab0-4775-9a34-874f5518b565","Type":"ContainerDied","Data":"b7ae2cf456a72320ee2d2adface7ec28e1a85c0942a156c384a0b841bb07af8a"} Jun 09 01:26:24.621224 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:24.621187 2555 generic.go:356] "Generic (PLEG): container finished" podID="5ebf50a7-9ab0-4775-9a34-874f5518b565" containerID="03e870c13f42a8b9d50c611d1c448cc6cd576c5f9c909ac3dccb1f883d25d13f" exitCode=0 Jun 09 01:26:24.621622 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:24.621273 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" event={"ID":"5ebf50a7-9ab0-4775-9a34-874f5518b565","Type":"ContainerDied","Data":"03e870c13f42a8b9d50c611d1c448cc6cd576c5f9c909ac3dccb1f883d25d13f"} Jun 09 01:26:25.768330 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:25.768306 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" Jun 09 01:26:25.836791 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:25.836746 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ebf50a7-9ab0-4775-9a34-874f5518b565-bundle\") pod \"5ebf50a7-9ab0-4775-9a34-874f5518b565\" (UID: \"5ebf50a7-9ab0-4775-9a34-874f5518b565\") " Jun 09 01:26:25.836969 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:25.836811 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2jr2\" (UniqueName: \"kubernetes.io/projected/5ebf50a7-9ab0-4775-9a34-874f5518b565-kube-api-access-h2jr2\") pod \"5ebf50a7-9ab0-4775-9a34-874f5518b565\" (UID: \"5ebf50a7-9ab0-4775-9a34-874f5518b565\") " Jun 09 01:26:25.836969 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:25.836867 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ebf50a7-9ab0-4775-9a34-874f5518b565-util\") pod \"5ebf50a7-9ab0-4775-9a34-874f5518b565\" (UID: \"5ebf50a7-9ab0-4775-9a34-874f5518b565\") " Jun 09 01:26:25.837220 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:25.837194 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ebf50a7-9ab0-4775-9a34-874f5518b565-bundle" (OuterVolumeSpecName: "bundle") pod "5ebf50a7-9ab0-4775-9a34-874f5518b565" (UID: "5ebf50a7-9ab0-4775-9a34-874f5518b565"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:26:25.838828 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:25.838806 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ebf50a7-9ab0-4775-9a34-874f5518b565-kube-api-access-h2jr2" (OuterVolumeSpecName: "kube-api-access-h2jr2") pod "5ebf50a7-9ab0-4775-9a34-874f5518b565" (UID: "5ebf50a7-9ab0-4775-9a34-874f5518b565"). InnerVolumeSpecName "kube-api-access-h2jr2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:26:25.842507 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:25.842471 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ebf50a7-9ab0-4775-9a34-874f5518b565-util" (OuterVolumeSpecName: "util") pod "5ebf50a7-9ab0-4775-9a34-874f5518b565" (UID: "5ebf50a7-9ab0-4775-9a34-874f5518b565"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 01:26:25.938536 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:25.938453 2555 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ebf50a7-9ab0-4775-9a34-874f5518b565-util\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:26:25.938536 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:25.938481 2555 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ebf50a7-9ab0-4775-9a34-874f5518b565-bundle\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:26:25.938536 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:25.938491 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2jr2\" (UniqueName: \"kubernetes.io/projected/5ebf50a7-9ab0-4775-9a34-874f5518b565-kube-api-access-h2jr2\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:26:26.634655 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:26.634620 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" Jun 09 01:26:26.634655 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:26.634637 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da7485hmlb" event={"ID":"5ebf50a7-9ab0-4775-9a34-874f5518b565","Type":"ContainerDied","Data":"c5cf03895da3c308eebab8ecf6d074f9e4c471e626432af490e1eb9147a4fa25"} Jun 09 01:26:26.634869 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:26:26.634687 2555 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5cf03895da3c308eebab8ecf6d074f9e4c471e626432af490e1eb9147a4fa25" Jun 09 01:27:20.053427 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.053395 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6fcd49ffbc-fsz7t"] Jun 09 01:27:20.053892 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.053841 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ebf50a7-9ab0-4775-9a34-874f5518b565" containerName="pull" Jun 09 01:27:20.053892 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.053853 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ebf50a7-9ab0-4775-9a34-874f5518b565" containerName="pull" Jun 09 01:27:20.053892 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.053864 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ebf50a7-9ab0-4775-9a34-874f5518b565" containerName="util" Jun 09 01:27:20.053892 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.053869 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ebf50a7-9ab0-4775-9a34-874f5518b565" containerName="util" Jun 09 01:27:20.053892 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.053876 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ebf50a7-9ab0-4775-9a34-874f5518b565" containerName="extract" Jun 09 01:27:20.053892 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.053881 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ebf50a7-9ab0-4775-9a34-874f5518b565" containerName="extract" Jun 09 01:27:20.054091 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.053947 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ebf50a7-9ab0-4775-9a34-874f5518b565" containerName="extract" Jun 09 01:27:20.057165 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.057145 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" Jun 09 01:27:20.059973 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.059947 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 09 01:27:20.060136 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.060046 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 09 01:27:20.060136 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.060046 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-fhzw5\"" Jun 09 01:27:20.070574 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.070519 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fcd49ffbc-fsz7t"] Jun 09 01:27:20.136953 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.136918 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lqm5\" (UniqueName: \"kubernetes.io/projected/cda4015b-f938-4938-910c-b87c3434fe52-kube-api-access-4lqm5\") pod \"maas-controller-6fcd49ffbc-fsz7t\" (UID: \"cda4015b-f938-4938-910c-b87c3434fe52\") " pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" Jun 09 01:27:20.137148 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.136963 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cda4015b-f938-4938-910c-b87c3434fe52-webhook-cert\") pod \"maas-controller-6fcd49ffbc-fsz7t\" (UID: \"cda4015b-f938-4938-910c-b87c3434fe52\") " pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" Jun 09 01:27:20.213082 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.213049 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5d888fff7f-68pvb"] Jun 09 01:27:20.216708 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.216693 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d888fff7f-68pvb" Jun 09 01:27:20.231172 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.231140 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d888fff7f-68pvb"] Jun 09 01:27:20.238045 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.238021 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lqm5\" (UniqueName: \"kubernetes.io/projected/cda4015b-f938-4938-910c-b87c3434fe52-kube-api-access-4lqm5\") pod \"maas-controller-6fcd49ffbc-fsz7t\" (UID: \"cda4015b-f938-4938-910c-b87c3434fe52\") " pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" Jun 09 01:27:20.238177 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.238062 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cda4015b-f938-4938-910c-b87c3434fe52-webhook-cert\") pod \"maas-controller-6fcd49ffbc-fsz7t\" (UID: \"cda4015b-f938-4938-910c-b87c3434fe52\") " pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" Jun 09 01:27:20.238219 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:27:20.238197 2555 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jun 09 01:27:20.238272 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:27:20.238261 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cda4015b-f938-4938-910c-b87c3434fe52-webhook-cert podName:cda4015b-f938-4938-910c-b87c3434fe52 nodeName:}" failed. No retries permitted until 2026-06-09 01:27:20.738242174 +0000 UTC m=+701.756546352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/cda4015b-f938-4938-910c-b87c3434fe52-webhook-cert") pod "maas-controller-6fcd49ffbc-fsz7t" (UID: "cda4015b-f938-4938-910c-b87c3434fe52") : secret "maas-controller-webhook-cert" not found Jun 09 01:27:20.248958 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.248932 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lqm5\" (UniqueName: \"kubernetes.io/projected/cda4015b-f938-4938-910c-b87c3434fe52-kube-api-access-4lqm5\") pod \"maas-controller-6fcd49ffbc-fsz7t\" (UID: \"cda4015b-f938-4938-910c-b87c3434fe52\") " pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" Jun 09 01:27:20.339612 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.339476 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62eb8970-32af-4bc9-ac3f-80d652a7ea78-webhook-cert\") pod \"maas-controller-5d888fff7f-68pvb\" (UID: \"62eb8970-32af-4bc9-ac3f-80d652a7ea78\") " pod="opendatahub/maas-controller-5d888fff7f-68pvb" Jun 09 01:27:20.339787 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.339630 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjwxp\" (UniqueName: \"kubernetes.io/projected/62eb8970-32af-4bc9-ac3f-80d652a7ea78-kube-api-access-pjwxp\") pod \"maas-controller-5d888fff7f-68pvb\" (UID: \"62eb8970-32af-4bc9-ac3f-80d652a7ea78\") " pod="opendatahub/maas-controller-5d888fff7f-68pvb" Jun 09 01:27:20.440296 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.440259 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62eb8970-32af-4bc9-ac3f-80d652a7ea78-webhook-cert\") pod \"maas-controller-5d888fff7f-68pvb\" (UID: \"62eb8970-32af-4bc9-ac3f-80d652a7ea78\") " pod="opendatahub/maas-controller-5d888fff7f-68pvb" Jun 09 01:27:20.440486 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.440421 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjwxp\" (UniqueName: \"kubernetes.io/projected/62eb8970-32af-4bc9-ac3f-80d652a7ea78-kube-api-access-pjwxp\") pod \"maas-controller-5d888fff7f-68pvb\" (UID: \"62eb8970-32af-4bc9-ac3f-80d652a7ea78\") " pod="opendatahub/maas-controller-5d888fff7f-68pvb" Jun 09 01:27:20.442763 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.442737 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62eb8970-32af-4bc9-ac3f-80d652a7ea78-webhook-cert\") pod \"maas-controller-5d888fff7f-68pvb\" (UID: \"62eb8970-32af-4bc9-ac3f-80d652a7ea78\") " pod="opendatahub/maas-controller-5d888fff7f-68pvb" Jun 09 01:27:20.454417 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.454390 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjwxp\" (UniqueName: \"kubernetes.io/projected/62eb8970-32af-4bc9-ac3f-80d652a7ea78-kube-api-access-pjwxp\") pod \"maas-controller-5d888fff7f-68pvb\" (UID: \"62eb8970-32af-4bc9-ac3f-80d652a7ea78\") " pod="opendatahub/maas-controller-5d888fff7f-68pvb" Jun 09 01:27:20.527094 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.527058 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d888fff7f-68pvb" Jun 09 01:27:20.662574 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.662534 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d888fff7f-68pvb"] Jun 09 01:27:20.662743 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:27:20.662721 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62eb8970_32af_4bc9_ac3f_80d652a7ea78.slice/crio-9e7a6eaf4490f9677dc04f0dea8fb576630f0fd4ad8a9823355afa18f4fddc13 WatchSource:0}: Error finding container 9e7a6eaf4490f9677dc04f0dea8fb576630f0fd4ad8a9823355afa18f4fddc13: Status 404 returned error can't find the container with id 9e7a6eaf4490f9677dc04f0dea8fb576630f0fd4ad8a9823355afa18f4fddc13 Jun 09 01:27:20.663965 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.663947 2555 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 01:27:20.743182 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.743142 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cda4015b-f938-4938-910c-b87c3434fe52-webhook-cert\") pod \"maas-controller-6fcd49ffbc-fsz7t\" (UID: \"cda4015b-f938-4938-910c-b87c3434fe52\") " pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" Jun 09 01:27:20.745600 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.745573 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cda4015b-f938-4938-910c-b87c3434fe52-webhook-cert\") pod \"maas-controller-6fcd49ffbc-fsz7t\" (UID: \"cda4015b-f938-4938-910c-b87c3434fe52\") " pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" Jun 09 01:27:20.960625 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.960527 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d888fff7f-68pvb" event={"ID":"62eb8970-32af-4bc9-ac3f-80d652a7ea78","Type":"ContainerStarted","Data":"9e7a6eaf4490f9677dc04f0dea8fb576630f0fd4ad8a9823355afa18f4fddc13"} Jun 09 01:27:20.967877 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:20.967858 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" Jun 09 01:27:21.103576 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:21.103532 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fcd49ffbc-fsz7t"] Jun 09 01:27:21.104077 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:27:21.104047 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcda4015b_f938_4938_910c_b87c3434fe52.slice/crio-ddaf2dbce15e8f7fe97593bdb4b625336b008942c233046f8f7b4594a339458d WatchSource:0}: Error finding container ddaf2dbce15e8f7fe97593bdb4b625336b008942c233046f8f7b4594a339458d: Status 404 returned error can't find the container with id ddaf2dbce15e8f7fe97593bdb4b625336b008942c233046f8f7b4594a339458d Jun 09 01:27:21.974754 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:21.974696 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" event={"ID":"cda4015b-f938-4938-910c-b87c3434fe52","Type":"ContainerStarted","Data":"ddaf2dbce15e8f7fe97593bdb4b625336b008942c233046f8f7b4594a339458d"} Jun 09 01:27:23.991783 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:23.991748 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d888fff7f-68pvb" event={"ID":"62eb8970-32af-4bc9-ac3f-80d652a7ea78","Type":"ContainerStarted","Data":"46c22dd2d47ca59a70f63d8d88cafb40d3830594e5d2be1ed14ebad482676e1f"} Jun 09 01:27:23.992318 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:23.991859 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5d888fff7f-68pvb" Jun 09 01:27:23.993308 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:23.993284 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" event={"ID":"cda4015b-f938-4938-910c-b87c3434fe52","Type":"ContainerStarted","Data":"cf643ca520b8110283103c3fc1cc423024232f2fa88212a25a262c9b7ce100ae"} Jun 09 01:27:23.993413 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:23.993338 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" Jun 09 01:27:24.012618 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:24.012574 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5d888fff7f-68pvb" podStartSLOduration=1.436683446 podStartE2EDuration="4.012561174s" totalImagesPullingTime="2.575877728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:27:24.010338254 +0000 UTC m=+705.028642463" watchObservedRunningTime="2026-06-09 01:27:24.012561174 +0000 UTC m=+705.030865365" Jun 09 01:27:24.026056 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:24.026009 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" podStartSLOduration=1.885065977 podStartE2EDuration="4.025995583s" totalImagesPullingTime="2.140929606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:27:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:27:24.025221806 +0000 UTC m=+705.043526034" watchObservedRunningTime="2026-06-09 01:27:24.025995583 +0000 UTC m=+705.044299775" Jun 09 01:27:35.004380 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:35.004349 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" Jun 09 01:27:35.004790 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:35.004425 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5d888fff7f-68pvb" Jun 09 01:27:35.062939 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:35.062908 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fcd49ffbc-fsz7t"] Jun 09 01:27:35.063118 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:35.063075 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" podUID="cda4015b-f938-4938-910c-b87c3434fe52" containerName="manager" containerID="cri-o://cf643ca520b8110283103c3fc1cc423024232f2fa88212a25a262c9b7ce100ae" gracePeriod=30 Jun 09 01:27:35.310017 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:35.309992 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" Jun 09 01:27:35.485056 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:35.485018 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cda4015b-f938-4938-910c-b87c3434fe52-webhook-cert\") pod \"cda4015b-f938-4938-910c-b87c3434fe52\" (UID: \"cda4015b-f938-4938-910c-b87c3434fe52\") " Jun 09 01:27:35.485232 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:35.485082 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lqm5\" (UniqueName: \"kubernetes.io/projected/cda4015b-f938-4938-910c-b87c3434fe52-kube-api-access-4lqm5\") pod \"cda4015b-f938-4938-910c-b87c3434fe52\" (UID: \"cda4015b-f938-4938-910c-b87c3434fe52\") " Jun 09 01:27:35.487362 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:35.487322 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cda4015b-f938-4938-910c-b87c3434fe52-kube-api-access-4lqm5" (OuterVolumeSpecName: "kube-api-access-4lqm5") pod "cda4015b-f938-4938-910c-b87c3434fe52" (UID: "cda4015b-f938-4938-910c-b87c3434fe52"). InnerVolumeSpecName "kube-api-access-4lqm5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:27:35.487362 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:35.487326 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cda4015b-f938-4938-910c-b87c3434fe52-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "cda4015b-f938-4938-910c-b87c3434fe52" (UID: "cda4015b-f938-4938-910c-b87c3434fe52"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 01:27:35.585790 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:35.585715 2555 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cda4015b-f938-4938-910c-b87c3434fe52-webhook-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:27:35.585790 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:35.585740 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lqm5\" (UniqueName: \"kubernetes.io/projected/cda4015b-f938-4938-910c-b87c3434fe52-kube-api-access-4lqm5\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:27:36.065263 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:36.065226 2555 generic.go:356] "Generic (PLEG): container finished" podID="cda4015b-f938-4938-910c-b87c3434fe52" containerID="cf643ca520b8110283103c3fc1cc423024232f2fa88212a25a262c9b7ce100ae" exitCode=0 Jun 09 01:27:36.065718 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:36.065273 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" event={"ID":"cda4015b-f938-4938-910c-b87c3434fe52","Type":"ContainerDied","Data":"cf643ca520b8110283103c3fc1cc423024232f2fa88212a25a262c9b7ce100ae"} Jun 09 01:27:36.065718 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:36.065295 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" Jun 09 01:27:36.065718 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:36.065322 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fcd49ffbc-fsz7t" event={"ID":"cda4015b-f938-4938-910c-b87c3434fe52","Type":"ContainerDied","Data":"ddaf2dbce15e8f7fe97593bdb4b625336b008942c233046f8f7b4594a339458d"} Jun 09 01:27:36.065718 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:36.065353 2555 scope.go:117] "RemoveContainer" containerID="cf643ca520b8110283103c3fc1cc423024232f2fa88212a25a262c9b7ce100ae" Jun 09 01:27:36.075168 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:36.075150 2555 scope.go:117] "RemoveContainer" containerID="cf643ca520b8110283103c3fc1cc423024232f2fa88212a25a262c9b7ce100ae" Jun 09 01:27:36.075422 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:27:36.075402 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf643ca520b8110283103c3fc1cc423024232f2fa88212a25a262c9b7ce100ae\": container with ID starting with cf643ca520b8110283103c3fc1cc423024232f2fa88212a25a262c9b7ce100ae not found: ID does not exist" containerID="cf643ca520b8110283103c3fc1cc423024232f2fa88212a25a262c9b7ce100ae" Jun 09 01:27:36.075496 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:36.075430 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf643ca520b8110283103c3fc1cc423024232f2fa88212a25a262c9b7ce100ae"} err="failed to get container status \"cf643ca520b8110283103c3fc1cc423024232f2fa88212a25a262c9b7ce100ae\": rpc error: code = NotFound desc = could not find container \"cf643ca520b8110283103c3fc1cc423024232f2fa88212a25a262c9b7ce100ae\": container with ID starting with cf643ca520b8110283103c3fc1cc423024232f2fa88212a25a262c9b7ce100ae not found: ID does not exist" Jun 09 01:27:36.087266 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:36.087207 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fcd49ffbc-fsz7t"] Jun 09 01:27:36.091307 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:36.091284 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6fcd49ffbc-fsz7t"] Jun 09 01:27:37.417039 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:37.417007 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cda4015b-f938-4938-910c-b87c3434fe52" path="/var/lib/kubelet/pods/cda4015b-f938-4938-910c-b87c3434fe52/volumes" Jun 09 01:27:55.298835 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:55.298794 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["redhat-ai-gateway-infra/maas-api-774f96689f-gq79f"] Jun 09 01:27:55.299678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:55.299468 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cda4015b-f938-4938-910c-b87c3434fe52" containerName="manager" Jun 09 01:27:55.299678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:55.299489 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="cda4015b-f938-4938-910c-b87c3434fe52" containerName="manager" Jun 09 01:27:55.299678 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:55.299602 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="cda4015b-f938-4938-910c-b87c3434fe52" containerName="manager" Jun 09 01:27:55.301892 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:55.301871 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-774f96689f-gq79f" Jun 09 01:27:55.304641 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:55.304616 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jun 09 01:27:55.304775 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:55.304662 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"redhat-ai-gateway-infra\"/\"maas-api-dockercfg-vfg2s\"" Jun 09 01:27:55.317493 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:55.317470 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-774f96689f-gq79f"] Jun 09 01:27:55.362508 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:55.362478 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7wld\" (UniqueName: \"kubernetes.io/projected/be26b3ad-6f31-447b-81c4-8a481ac60697-kube-api-access-m7wld\") pod \"maas-api-774f96689f-gq79f\" (UID: \"be26b3ad-6f31-447b-81c4-8a481ac60697\") " pod="redhat-ai-gateway-infra/maas-api-774f96689f-gq79f" Jun 09 01:27:55.362749 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:55.362528 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be26b3ad-6f31-447b-81c4-8a481ac60697-maas-api-tls\") pod \"maas-api-774f96689f-gq79f\" (UID: \"be26b3ad-6f31-447b-81c4-8a481ac60697\") " pod="redhat-ai-gateway-infra/maas-api-774f96689f-gq79f" Jun 09 01:27:55.463913 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:55.463879 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7wld\" (UniqueName: \"kubernetes.io/projected/be26b3ad-6f31-447b-81c4-8a481ac60697-kube-api-access-m7wld\") pod \"maas-api-774f96689f-gq79f\" (UID: \"be26b3ad-6f31-447b-81c4-8a481ac60697\") " pod="redhat-ai-gateway-infra/maas-api-774f96689f-gq79f" Jun 09 01:27:55.464091 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:55.463951 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be26b3ad-6f31-447b-81c4-8a481ac60697-maas-api-tls\") pod \"maas-api-774f96689f-gq79f\" (UID: \"be26b3ad-6f31-447b-81c4-8a481ac60697\") " pod="redhat-ai-gateway-infra/maas-api-774f96689f-gq79f" Jun 09 01:27:55.464091 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:27:55.464084 2555 secret.go:189] Couldn't get secret redhat-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jun 09 01:27:55.464208 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:27:55.464150 2555 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be26b3ad-6f31-447b-81c4-8a481ac60697-maas-api-tls podName:be26b3ad-6f31-447b-81c4-8a481ac60697 nodeName:}" failed. No retries permitted until 2026-06-09 01:27:55.964130639 +0000 UTC m=+736.982434811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/be26b3ad-6f31-447b-81c4-8a481ac60697-maas-api-tls") pod "maas-api-774f96689f-gq79f" (UID: "be26b3ad-6f31-447b-81c4-8a481ac60697") : secret "maas-api-serving-cert" not found Jun 09 01:27:55.472810 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:55.472777 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7wld\" (UniqueName: \"kubernetes.io/projected/be26b3ad-6f31-447b-81c4-8a481ac60697-kube-api-access-m7wld\") pod \"maas-api-774f96689f-gq79f\" (UID: \"be26b3ad-6f31-447b-81c4-8a481ac60697\") " pod="redhat-ai-gateway-infra/maas-api-774f96689f-gq79f" Jun 09 01:27:55.969185 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:55.969151 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be26b3ad-6f31-447b-81c4-8a481ac60697-maas-api-tls\") pod \"maas-api-774f96689f-gq79f\" (UID: \"be26b3ad-6f31-447b-81c4-8a481ac60697\") " pod="redhat-ai-gateway-infra/maas-api-774f96689f-gq79f" Jun 09 01:27:55.971644 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:55.971618 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/be26b3ad-6f31-447b-81c4-8a481ac60697-maas-api-tls\") pod \"maas-api-774f96689f-gq79f\" (UID: \"be26b3ad-6f31-447b-81c4-8a481ac60697\") " pod="redhat-ai-gateway-infra/maas-api-774f96689f-gq79f" Jun 09 01:27:56.212070 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:56.212034 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="redhat-ai-gateway-infra/maas-api-774f96689f-gq79f" Jun 09 01:27:56.350818 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:56.350743 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["redhat-ai-gateway-infra/maas-api-774f96689f-gq79f"] Jun 09 01:27:56.353229 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:27:56.353197 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe26b3ad_6f31_447b_81c4_8a481ac60697.slice/crio-e5e6ec2c82d26dd378885da924ba261e1d19e02d93b59bc4dc3701497f1a3083 WatchSource:0}: Error finding container e5e6ec2c82d26dd378885da924ba261e1d19e02d93b59bc4dc3701497f1a3083: Status 404 returned error can't find the container with id e5e6ec2c82d26dd378885da924ba261e1d19e02d93b59bc4dc3701497f1a3083 Jun 09 01:27:57.197356 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:57.197315 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-774f96689f-gq79f" event={"ID":"be26b3ad-6f31-447b-81c4-8a481ac60697","Type":"ContainerStarted","Data":"e5e6ec2c82d26dd378885da924ba261e1d19e02d93b59bc4dc3701497f1a3083"} Jun 09 01:27:58.204717 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:58.204680 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="redhat-ai-gateway-infra/maas-api-774f96689f-gq79f" event={"ID":"be26b3ad-6f31-447b-81c4-8a481ac60697","Type":"ContainerStarted","Data":"ac80c3321bbe90d33d0575a4c5a2bbcd962c0efbb9c8be6fe5856c7a9b5e2952"} Jun 09 01:27:58.205083 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:58.204746 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="redhat-ai-gateway-infra/maas-api-774f96689f-gq79f" Jun 09 01:27:58.222805 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:27:58.222759 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="redhat-ai-gateway-infra/maas-api-774f96689f-gq79f" podStartSLOduration=1.497637308 podStartE2EDuration="3.222745306s" totalImagesPullingTime="1.725107998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:27:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:27:58.220806503 +0000 UTC m=+739.239110716" watchObservedRunningTime="2026-06-09 01:27:58.222745306 +0000 UTC m=+739.241049498" Jun 09 01:28:04.216753 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:04.216721 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="redhat-ai-gateway-infra/maas-api-774f96689f-gq79f" Jun 09 01:28:59.515992 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.515913 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8"] Jun 09 01:28:59.520572 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.520532 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.523904 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.523576 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 09 01:28:59.525005 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.524981 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 09 01:28:59.525128 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.525079 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 09 01:28:59.525467 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.525442 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xbzpq\"" Jun 09 01:28:59.530414 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.530393 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8"] Jun 09 01:28:59.664813 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.664776 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.664813 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.664812 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq9ft\" (UniqueName: \"kubernetes.io/projected/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-kube-api-access-fq9ft\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.665067 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.664838 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.665067 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.664862 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.665067 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.664961 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.665199 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.665079 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.765837 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.765795 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.766080 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.765866 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.766080 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.765889 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq9ft\" (UniqueName: \"kubernetes.io/projected/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-kube-api-access-fq9ft\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.766080 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.765919 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.766080 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.765939 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.766080 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.765999 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.766362 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.766332 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.766422 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.766373 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.766474 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.766430 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.768140 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.768117 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.768387 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.768368 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.774216 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.774193 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq9ft\" (UniqueName: \"kubernetes.io/projected/4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54-kube-api-access-fq9ft\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8\" (UID: \"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.833587 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.833521 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:28:59.973179 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:28:59.973148 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8"] Jun 09 01:28:59.974570 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:28:59.974520 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dfd0a73_9d1a_4ed6_b920_5c8898eb8d54.slice/crio-6572e4612303c26ca23c8d78b361f933ed299d47834f2a91ec4677ccee99431e WatchSource:0}: Error finding container 6572e4612303c26ca23c8d78b361f933ed299d47834f2a91ec4677ccee99431e: Status 404 returned error can't find the container with id 6572e4612303c26ca23c8d78b361f933ed299d47834f2a91ec4677ccee99431e Jun 09 01:29:00.577632 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:00.577595 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" event={"ID":"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54","Type":"ContainerStarted","Data":"6572e4612303c26ca23c8d78b361f933ed299d47834f2a91ec4677ccee99431e"} Jun 09 01:29:04.616867 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:04.616824 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" event={"ID":"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54","Type":"ContainerStarted","Data":"b2fa4756ab14937073f53979ddf73e5629f3022d8e44e96658e50bf99fc8fa95"} Jun 09 01:29:05.624331 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:05.624298 2555 generic.go:356] "Generic (PLEG): container finished" podID="4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54" containerID="b2fa4756ab14937073f53979ddf73e5629f3022d8e44e96658e50bf99fc8fa95" exitCode=0 Jun 09 01:29:05.624803 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:05.624378 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" event={"ID":"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54","Type":"ContainerDied","Data":"b2fa4756ab14937073f53979ddf73e5629f3022d8e44e96658e50bf99fc8fa95"} Jun 09 01:29:07.638842 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:07.638799 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" event={"ID":"4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54","Type":"ContainerStarted","Data":"2bb428646acd98ced0419141c8b9807c068471b81a38bb7495005c714b6bfeaa"} Jun 09 01:29:07.657773 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:07.657725 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" podStartSLOduration=2.680782655 podStartE2EDuration="8.657710568s" totalImagesPullingTime="4.716098253s" totalInitContainerRuntime="1.26082966s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:28:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:29:07.655777078 +0000 UTC m=+808.674081271" watchObservedRunningTime="2026-06-09 01:29:07.657710568 +0000 UTC m=+808.676014759" Jun 09 01:29:07.904064 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:07.903985 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs"] Jun 09 01:29:07.908133 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:07.908110 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:07.912012 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:07.911984 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 09 01:29:07.918453 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:07.918427 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs"] Jun 09 01:29:08.048720 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.048677 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19fc165a-0d06-4822-a3e4-53826fd8617b-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.048898 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.048788 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xshk\" (UniqueName: \"kubernetes.io/projected/19fc165a-0d06-4822-a3e4-53826fd8617b-kube-api-access-5xshk\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.048898 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.048841 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19fc165a-0d06-4822-a3e4-53826fd8617b-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.048898 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.048892 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fc165a-0d06-4822-a3e4-53826fd8617b-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.049048 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.048926 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fc165a-0d06-4822-a3e4-53826fd8617b-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.049048 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.049005 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/19fc165a-0d06-4822-a3e4-53826fd8617b-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.150053 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.150019 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fc165a-0d06-4822-a3e4-53826fd8617b-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.150053 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.150053 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fc165a-0d06-4822-a3e4-53826fd8617b-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.150305 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.150082 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/19fc165a-0d06-4822-a3e4-53826fd8617b-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.150305 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.150104 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19fc165a-0d06-4822-a3e4-53826fd8617b-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.150305 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.150171 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xshk\" (UniqueName: \"kubernetes.io/projected/19fc165a-0d06-4822-a3e4-53826fd8617b-kube-api-access-5xshk\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.150305 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.150220 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19fc165a-0d06-4822-a3e4-53826fd8617b-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.150519 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.150493 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19fc165a-0d06-4822-a3e4-53826fd8617b-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.150626 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.150579 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/19fc165a-0d06-4822-a3e4-53826fd8617b-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.150702 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.150686 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19fc165a-0d06-4822-a3e4-53826fd8617b-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.152938 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.152912 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19fc165a-0d06-4822-a3e4-53826fd8617b-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.153097 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.153080 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19fc165a-0d06-4822-a3e4-53826fd8617b-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.157604 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.157533 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xshk\" (UniqueName: \"kubernetes.io/projected/19fc165a-0d06-4822-a3e4-53826fd8617b-kube-api-access-5xshk\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-jmfhs\" (UID: \"19fc165a-0d06-4822-a3e4-53826fd8617b\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.219658 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.219623 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:08.360920 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.360874 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs"] Jun 09 01:29:08.361594 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:29:08.361565 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19fc165a_0d06_4822_a3e4_53826fd8617b.slice/crio-a638be071d385599a983223a34cfabed73ae0bafd5391c15775c32ccd3e99883 WatchSource:0}: Error finding container a638be071d385599a983223a34cfabed73ae0bafd5391c15775c32ccd3e99883: Status 404 returned error can't find the container with id a638be071d385599a983223a34cfabed73ae0bafd5391c15775c32ccd3e99883 Jun 09 01:29:08.649233 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.649186 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" event={"ID":"19fc165a-0d06-4822-a3e4-53826fd8617b","Type":"ContainerStarted","Data":"9709ef10ea2d3e4d0e9b034afe7b7a202756dd1d6b19a41bc303c69864eec67c"} Jun 09 01:29:08.649693 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:08.649259 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" event={"ID":"19fc165a-0d06-4822-a3e4-53826fd8617b","Type":"ContainerStarted","Data":"a638be071d385599a983223a34cfabed73ae0bafd5391c15775c32ccd3e99883"} Jun 09 01:29:09.833733 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:09.833686 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:29:09.834159 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:09.833769 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:29:09.845773 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:09.845749 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:29:10.663466 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:10.663432 2555 generic.go:356] "Generic (PLEG): container finished" podID="19fc165a-0d06-4822-a3e4-53826fd8617b" containerID="9709ef10ea2d3e4d0e9b034afe7b7a202756dd1d6b19a41bc303c69864eec67c" exitCode=0 Jun 09 01:29:10.663653 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:10.663506 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" event={"ID":"19fc165a-0d06-4822-a3e4-53826fd8617b","Type":"ContainerDied","Data":"9709ef10ea2d3e4d0e9b034afe7b7a202756dd1d6b19a41bc303c69864eec67c"} Jun 09 01:29:10.674717 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:10.674696 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8" Jun 09 01:29:11.671430 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:11.671394 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" event={"ID":"19fc165a-0d06-4822-a3e4-53826fd8617b","Type":"ContainerStarted","Data":"8d8928f4dd0b371eea2a43f98bc272c8a835270e13a762e274723962f4b63c56"} Jun 09 01:29:11.691669 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:11.691615 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" podStartSLOduration=3.263857088 podStartE2EDuration="4.691600668s" totalImagesPullingTime="182.625142ms" totalInitContainerRuntime="1.245118438s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:29:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:29:11.688799472 +0000 UTC m=+812.707103667" watchObservedRunningTime="2026-06-09 01:29:11.691600668 +0000 UTC m=+812.709904916" Jun 09 01:29:18.220323 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:18.220287 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:18.220720 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:18.220359 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:18.232501 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:18.232478 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:18.726466 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:18.726441 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-jmfhs" Jun 09 01:29:27.117506 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.117472 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw"] Jun 09 01:29:27.129024 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.128999 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw"] Jun 09 01:29:27.129196 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.129113 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.131904 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.131878 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 09 01:29:27.221950 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.221918 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c72ea3e1-4e09-4e9c-b574-85d711da0d90-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.222151 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.221983 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c72ea3e1-4e09-4e9c-b574-85d711da0d90-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.222151 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.222026 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c72ea3e1-4e09-4e9c-b574-85d711da0d90-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.222151 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.222061 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69w7c\" (UniqueName: \"kubernetes.io/projected/c72ea3e1-4e09-4e9c-b574-85d711da0d90-kube-api-access-69w7c\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.222151 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.222105 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c72ea3e1-4e09-4e9c-b574-85d711da0d90-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.222151 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.222123 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c72ea3e1-4e09-4e9c-b574-85d711da0d90-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.322897 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.322857 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c72ea3e1-4e09-4e9c-b574-85d711da0d90-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.323098 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.322926 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c72ea3e1-4e09-4e9c-b574-85d711da0d90-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.323098 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.322955 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c72ea3e1-4e09-4e9c-b574-85d711da0d90-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.323098 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.323047 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69w7c\" (UniqueName: \"kubernetes.io/projected/c72ea3e1-4e09-4e9c-b574-85d711da0d90-kube-api-access-69w7c\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.323098 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.323082 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c72ea3e1-4e09-4e9c-b574-85d711da0d90-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.323322 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.323122 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c72ea3e1-4e09-4e9c-b574-85d711da0d90-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.323409 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.323385 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c72ea3e1-4e09-4e9c-b574-85d711da0d90-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.323499 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.323460 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c72ea3e1-4e09-4e9c-b574-85d711da0d90-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.323577 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.323526 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c72ea3e1-4e09-4e9c-b574-85d711da0d90-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.325180 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.325162 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c72ea3e1-4e09-4e9c-b574-85d711da0d90-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.325552 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.325523 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c72ea3e1-4e09-4e9c-b574-85d711da0d90-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.330959 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.330939 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69w7c\" (UniqueName: \"kubernetes.io/projected/c72ea3e1-4e09-4e9c-b574-85d711da0d90-kube-api-access-69w7c\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw\" (UID: \"c72ea3e1-4e09-4e9c-b574-85d711da0d90\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.440741 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.440659 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:29:27.580621 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.580589 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw"] Jun 09 01:29:27.581260 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:29:27.581233 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc72ea3e1_4e09_4e9c_b574_85d711da0d90.slice/crio-3b7bb28bfa8af700aff87db809619befa7276a13bf87e1ac8c52d534e3f5d804 WatchSource:0}: Error finding container 3b7bb28bfa8af700aff87db809619befa7276a13bf87e1ac8c52d534e3f5d804: Status 404 returned error can't find the container with id 3b7bb28bfa8af700aff87db809619befa7276a13bf87e1ac8c52d534e3f5d804 Jun 09 01:29:27.770845 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.770753 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" event={"ID":"c72ea3e1-4e09-4e9c-b574-85d711da0d90","Type":"ContainerStarted","Data":"1690eac59a3845a47bd0cb96e995419c244462065c8f4959c4d8d4a374ca5abd"} Jun 09 01:29:27.770845 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:27.770819 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" event={"ID":"c72ea3e1-4e09-4e9c-b574-85d711da0d90","Type":"ContainerStarted","Data":"3b7bb28bfa8af700aff87db809619befa7276a13bf87e1ac8c52d534e3f5d804"} Jun 09 01:29:48.190615 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.190582 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f"] Jun 09 01:29:48.197733 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.197709 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.201151 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.201123 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 09 01:29:48.203872 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.203846 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f"] Jun 09 01:29:48.332220 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.332190 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/532418cf-01a5-4897-a4b3-c51a05e5e894-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.332453 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.332228 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/532418cf-01a5-4897-a4b3-c51a05e5e894-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.332453 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.332262 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/532418cf-01a5-4897-a4b3-c51a05e5e894-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.332453 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.332281 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cthnq\" (UniqueName: \"kubernetes.io/projected/532418cf-01a5-4897-a4b3-c51a05e5e894-kube-api-access-cthnq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.332453 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.332322 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/532418cf-01a5-4897-a4b3-c51a05e5e894-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.332453 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.332389 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/532418cf-01a5-4897-a4b3-c51a05e5e894-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.432927 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.432880 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/532418cf-01a5-4897-a4b3-c51a05e5e894-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.433129 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.432949 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/532418cf-01a5-4897-a4b3-c51a05e5e894-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.433129 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.432984 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/532418cf-01a5-4897-a4b3-c51a05e5e894-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.433254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.433126 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/532418cf-01a5-4897-a4b3-c51a05e5e894-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.433254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.433162 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cthnq\" (UniqueName: \"kubernetes.io/projected/532418cf-01a5-4897-a4b3-c51a05e5e894-kube-api-access-cthnq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.433254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.433218 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/532418cf-01a5-4897-a4b3-c51a05e5e894-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.433410 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.433252 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/532418cf-01a5-4897-a4b3-c51a05e5e894-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.433410 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.433316 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/532418cf-01a5-4897-a4b3-c51a05e5e894-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.433584 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.433530 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/532418cf-01a5-4897-a4b3-c51a05e5e894-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.435253 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.435233 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/532418cf-01a5-4897-a4b3-c51a05e5e894-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.435496 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.435481 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/532418cf-01a5-4897-a4b3-c51a05e5e894-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.441318 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.441255 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cthnq\" (UniqueName: \"kubernetes.io/projected/532418cf-01a5-4897-a4b3-c51a05e5e894-kube-api-access-cthnq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f\" (UID: \"532418cf-01a5-4897-a4b3-c51a05e5e894\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.510555 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.510515 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:48.648197 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:29:48.648163 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod532418cf_01a5_4897_a4b3_c51a05e5e894.slice/crio-95892be3d9adf6a677319c7902de3ea5d23018d403bc1a7023461c6c1214314d WatchSource:0}: Error finding container 95892be3d9adf6a677319c7902de3ea5d23018d403bc1a7023461c6c1214314d: Status 404 returned error can't find the container with id 95892be3d9adf6a677319c7902de3ea5d23018d403bc1a7023461c6c1214314d Jun 09 01:29:48.649527 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.649496 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f"] Jun 09 01:29:48.901962 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.901925 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" event={"ID":"532418cf-01a5-4897-a4b3-c51a05e5e894","Type":"ContainerStarted","Data":"3abc9eb0249bb30ce122719d801512ab5252ca63a159e9e3d6ae18705f3cad51"} Jun 09 01:29:48.902140 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:48.901976 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" event={"ID":"532418cf-01a5-4897-a4b3-c51a05e5e894","Type":"ContainerStarted","Data":"95892be3d9adf6a677319c7902de3ea5d23018d403bc1a7023461c6c1214314d"} Jun 09 01:29:50.915271 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:50.915230 2555 generic.go:356] "Generic (PLEG): container finished" podID="532418cf-01a5-4897-a4b3-c51a05e5e894" containerID="3abc9eb0249bb30ce122719d801512ab5252ca63a159e9e3d6ae18705f3cad51" exitCode=0 Jun 09 01:29:50.915689 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:50.915302 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" event={"ID":"532418cf-01a5-4897-a4b3-c51a05e5e894","Type":"ContainerDied","Data":"3abc9eb0249bb30ce122719d801512ab5252ca63a159e9e3d6ae18705f3cad51"} Jun 09 01:29:51.922768 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:51.922729 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" event={"ID":"532418cf-01a5-4897-a4b3-c51a05e5e894","Type":"ContainerStarted","Data":"001fd8a1214bbb667679997d39a30500646f508381cb6061267a96f747bcfe2a"} Jun 09 01:29:51.944688 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:51.944642 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" podStartSLOduration=2.488813665 podStartE2EDuration="3.944627766s" totalImagesPullingTime="198.468821ms" totalInitContainerRuntime="1.25734528s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:29:51.942829182 +0000 UTC m=+852.961133397" watchObservedRunningTime="2026-06-09 01:29:51.944627766 +0000 UTC m=+852.962931957" Jun 09 01:29:58.511670 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:58.511630 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:58.512176 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:58.511696 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:58.523963 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:58.523937 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:29:58.981152 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:29:58.981127 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f" Jun 09 01:30:05.804383 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:05.804350 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r"] Jun 09 01:30:05.809910 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:05.809882 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:05.812766 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:05.812737 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 09 01:30:05.817873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:05.817847 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r"] Jun 09 01:30:05.906658 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:05.906618 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b66e01d3-cf61-40df-b048-10fdd64557f6-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:05.906866 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:05.906715 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b66e01d3-cf61-40df-b048-10fdd64557f6-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:05.906866 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:05.906773 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b66e01d3-cf61-40df-b048-10fdd64557f6-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:05.906866 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:05.906796 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tvwt\" (UniqueName: \"kubernetes.io/projected/b66e01d3-cf61-40df-b048-10fdd64557f6-kube-api-access-9tvwt\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:05.906866 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:05.906838 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b66e01d3-cf61-40df-b048-10fdd64557f6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:05.907017 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:05.906886 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b66e01d3-cf61-40df-b048-10fdd64557f6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:06.007601 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:06.007535 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b66e01d3-cf61-40df-b048-10fdd64557f6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:06.007789 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:06.007621 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b66e01d3-cf61-40df-b048-10fdd64557f6-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:06.007789 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:06.007679 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b66e01d3-cf61-40df-b048-10fdd64557f6-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:06.007789 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:06.007723 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b66e01d3-cf61-40df-b048-10fdd64557f6-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:06.007789 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:06.007740 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tvwt\" (UniqueName: \"kubernetes.io/projected/b66e01d3-cf61-40df-b048-10fdd64557f6-kube-api-access-9tvwt\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:06.007990 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:06.007878 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b66e01d3-cf61-40df-b048-10fdd64557f6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:06.008088 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:06.008067 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b66e01d3-cf61-40df-b048-10fdd64557f6-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:06.008177 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:06.008157 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b66e01d3-cf61-40df-b048-10fdd64557f6-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:06.008272 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:06.008251 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b66e01d3-cf61-40df-b048-10fdd64557f6-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:06.010377 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:06.010358 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b66e01d3-cf61-40df-b048-10fdd64557f6-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:06.010671 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:06.010654 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b66e01d3-cf61-40df-b048-10fdd64557f6-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:06.015775 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:06.015748 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tvwt\" (UniqueName: \"kubernetes.io/projected/b66e01d3-cf61-40df-b048-10fdd64557f6-kube-api-access-9tvwt\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r\" (UID: \"b66e01d3-cf61-40df-b048-10fdd64557f6\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:06.122839 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:06.122745 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:30:06.263599 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:06.263566 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r"] Jun 09 01:30:06.264083 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:30:06.264057 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb66e01d3_cf61_40df_b048_10fdd64557f6.slice/crio-b6b6a339d56fb9877bb81d077e4c76b580653c5f012178f0300c3ecae0546f04 WatchSource:0}: Error finding container b6b6a339d56fb9877bb81d077e4c76b580653c5f012178f0300c3ecae0546f04: Status 404 returned error can't find the container with id b6b6a339d56fb9877bb81d077e4c76b580653c5f012178f0300c3ecae0546f04 Jun 09 01:30:07.022438 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:07.022406 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" event={"ID":"b66e01d3-cf61-40df-b048-10fdd64557f6","Type":"ContainerStarted","Data":"77d425004d3217343e11000af30fe2d4971fda89e8c21e5a2ad098ca4d532d1a"} Jun 09 01:30:07.022908 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:07.022469 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" event={"ID":"b66e01d3-cf61-40df-b048-10fdd64557f6","Type":"ContainerStarted","Data":"b6b6a339d56fb9877bb81d077e4c76b580653c5f012178f0300c3ecae0546f04"} Jun 09 01:30:32.180532 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:32.180492 2555 generic.go:356] "Generic (PLEG): container finished" podID="c72ea3e1-4e09-4e9c-b574-85d711da0d90" containerID="1690eac59a3845a47bd0cb96e995419c244462065c8f4959c4d8d4a374ca5abd" exitCode=0 Jun 09 01:30:32.180966 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:32.180565 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" event={"ID":"c72ea3e1-4e09-4e9c-b574-85d711da0d90","Type":"ContainerDied","Data":"1690eac59a3845a47bd0cb96e995419c244462065c8f4959c4d8d4a374ca5abd"} Jun 09 01:30:33.189251 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:33.189215 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" event={"ID":"c72ea3e1-4e09-4e9c-b574-85d711da0d90","Type":"ContainerStarted","Data":"86c5c735f70e4f75042f129002714beacc72580fa4276784b15bf0fed1f5a410"} Jun 09 01:30:33.211502 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:33.211451 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" podStartSLOduration=2.14959016 podStartE2EDuration="1m6.211434236s" totalImagesPullingTime="192.994107ms" totalInitContainerRuntime="1m3.868849969s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:29:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:30:33.21034438 +0000 UTC m=+894.228648590" watchObservedRunningTime="2026-06-09 01:30:33.211434236 +0000 UTC m=+894.229738450" Jun 09 01:30:37.441063 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:37.441031 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:30:37.441585 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:37.441098 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:30:37.453698 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:37.453674 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:30:38.237432 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:38.237399 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw" Jun 09 01:30:39.454588 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:39.454464 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:30:39.461873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:39.458374 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:30:39.461873 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:39.458559 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:30:39.462455 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:39.462437 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:30:58.349621 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:58.349579 2555 generic.go:356] "Generic (PLEG): container finished" podID="b66e01d3-cf61-40df-b048-10fdd64557f6" containerID="77d425004d3217343e11000af30fe2d4971fda89e8c21e5a2ad098ca4d532d1a" exitCode=0 Jun 09 01:30:58.349621 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:58.349622 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" event={"ID":"b66e01d3-cf61-40df-b048-10fdd64557f6","Type":"ContainerDied","Data":"77d425004d3217343e11000af30fe2d4971fda89e8c21e5a2ad098ca4d532d1a"} Jun 09 01:30:59.358248 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:59.358211 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" event={"ID":"b66e01d3-cf61-40df-b048-10fdd64557f6","Type":"ContainerStarted","Data":"5c99fe536bbaa0238747707bad01497d2a361932af6096348eb8382b1c85dfc4"} Jun 09 01:30:59.379701 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:30:59.379648 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" podStartSLOduration=2.81369108 podStartE2EDuration="54.379633816s" totalImagesPullingTime="189.634306ms" totalInitContainerRuntime="51.37630843s" isStatefulPod=false podCreationTimestamp="2026-06-09 01:30:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:30:59.377412875 +0000 UTC m=+920.395717068" watchObservedRunningTime="2026-06-09 01:30:59.379633816 +0000 UTC m=+920.397938007" Jun 09 01:31:06.123297 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:31:06.123248 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:31:06.123833 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:31:06.123367 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:31:06.135819 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:31:06.135795 2555 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:31:06.418177 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:31:06.418089 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r" Jun 09 01:32:47.192216 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:47.192180 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d888fff7f-68pvb"] Jun 09 01:32:47.192753 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:47.192435 2555 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5d888fff7f-68pvb" podUID="62eb8970-32af-4bc9-ac3f-80d652a7ea78" containerName="manager" containerID="cri-o://46c22dd2d47ca59a70f63d8d88cafb40d3830594e5d2be1ed14ebad482676e1f" gracePeriod=30 Jun 09 01:32:47.454206 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:47.454144 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d888fff7f-68pvb" Jun 09 01:32:47.516232 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:47.516197 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjwxp\" (UniqueName: \"kubernetes.io/projected/62eb8970-32af-4bc9-ac3f-80d652a7ea78-kube-api-access-pjwxp\") pod \"62eb8970-32af-4bc9-ac3f-80d652a7ea78\" (UID: \"62eb8970-32af-4bc9-ac3f-80d652a7ea78\") " Jun 09 01:32:47.516399 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:47.516297 2555 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62eb8970-32af-4bc9-ac3f-80d652a7ea78-webhook-cert\") pod \"62eb8970-32af-4bc9-ac3f-80d652a7ea78\" (UID: \"62eb8970-32af-4bc9-ac3f-80d652a7ea78\") " Jun 09 01:32:47.518403 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:47.518375 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62eb8970-32af-4bc9-ac3f-80d652a7ea78-kube-api-access-pjwxp" (OuterVolumeSpecName: "kube-api-access-pjwxp") pod "62eb8970-32af-4bc9-ac3f-80d652a7ea78" (UID: "62eb8970-32af-4bc9-ac3f-80d652a7ea78"). InnerVolumeSpecName "kube-api-access-pjwxp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 01:32:47.518403 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:47.518382 2555 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62eb8970-32af-4bc9-ac3f-80d652a7ea78-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "62eb8970-32af-4bc9-ac3f-80d652a7ea78" (UID: "62eb8970-32af-4bc9-ac3f-80d652a7ea78"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 01:32:47.617428 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:47.617392 2555 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/62eb8970-32af-4bc9-ac3f-80d652a7ea78-webhook-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:32:47.617428 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:47.617421 2555 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjwxp\" (UniqueName: \"kubernetes.io/projected/62eb8970-32af-4bc9-ac3f-80d652a7ea78-kube-api-access-pjwxp\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\"" Jun 09 01:32:48.049206 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:48.049170 2555 generic.go:356] "Generic (PLEG): container finished" podID="62eb8970-32af-4bc9-ac3f-80d652a7ea78" containerID="46c22dd2d47ca59a70f63d8d88cafb40d3830594e5d2be1ed14ebad482676e1f" exitCode=0 Jun 09 01:32:48.049406 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:48.049220 2555 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d888fff7f-68pvb" Jun 09 01:32:48.049406 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:48.049261 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d888fff7f-68pvb" event={"ID":"62eb8970-32af-4bc9-ac3f-80d652a7ea78","Type":"ContainerDied","Data":"46c22dd2d47ca59a70f63d8d88cafb40d3830594e5d2be1ed14ebad482676e1f"} Jun 09 01:32:48.049406 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:48.049314 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d888fff7f-68pvb" event={"ID":"62eb8970-32af-4bc9-ac3f-80d652a7ea78","Type":"ContainerDied","Data":"9e7a6eaf4490f9677dc04f0dea8fb576630f0fd4ad8a9823355afa18f4fddc13"} Jun 09 01:32:48.049406 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:48.049336 2555 scope.go:117] "RemoveContainer" containerID="46c22dd2d47ca59a70f63d8d88cafb40d3830594e5d2be1ed14ebad482676e1f" Jun 09 01:32:48.059813 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:48.059796 2555 scope.go:117] "RemoveContainer" containerID="46c22dd2d47ca59a70f63d8d88cafb40d3830594e5d2be1ed14ebad482676e1f" Jun 09 01:32:48.060062 ip-10-0-143-161 kubenswrapper[2555]: E0609 01:32:48.060047 2555 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46c22dd2d47ca59a70f63d8d88cafb40d3830594e5d2be1ed14ebad482676e1f\": container with ID starting with 46c22dd2d47ca59a70f63d8d88cafb40d3830594e5d2be1ed14ebad482676e1f not found: ID does not exist" containerID="46c22dd2d47ca59a70f63d8d88cafb40d3830594e5d2be1ed14ebad482676e1f" Jun 09 01:32:48.060104 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:48.060071 2555 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46c22dd2d47ca59a70f63d8d88cafb40d3830594e5d2be1ed14ebad482676e1f"} err="failed to get container status \"46c22dd2d47ca59a70f63d8d88cafb40d3830594e5d2be1ed14ebad482676e1f\": rpc error: code = NotFound desc = could not find container \"46c22dd2d47ca59a70f63d8d88cafb40d3830594e5d2be1ed14ebad482676e1f\": container with ID starting with 46c22dd2d47ca59a70f63d8d88cafb40d3830594e5d2be1ed14ebad482676e1f not found: ID does not exist" Jun 09 01:32:48.071899 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:48.071875 2555 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d888fff7f-68pvb"] Jun 09 01:32:48.075354 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:48.075331 2555 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5d888fff7f-68pvb"] Jun 09 01:32:49.416879 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:32:49.416847 2555 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62eb8970-32af-4bc9-ac3f-80d652a7ea78" path="/var/lib/kubelet/pods/62eb8970-32af-4bc9-ac3f-80d652a7ea78/volumes" Jun 09 01:35:39.499564 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:35:39.499444 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:35:39.503657 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:35:39.503383 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:35:39.505202 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:35:39.505183 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:35:39.509400 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:35:39.509382 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:40:39.541948 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:40:39.541915 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:40:39.545992 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:40:39.545960 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:40:39.549130 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:40:39.549107 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:40:39.553834 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:40:39.553817 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:45:39.584072 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:45:39.583964 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:45:39.590105 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:45:39.588480 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:45:39.593989 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:45:39.593973 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:45:39.597846 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:45:39.597829 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:50:39.631591 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:50:39.631477 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:50:39.636231 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:50:39.636211 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:50:39.641563 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:50:39.641524 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:50:39.645684 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:50:39.645668 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-gpkxb_a3a9620d-b3b8-48ae-b007-acd921f2aeac/ovn-acl-logging/0.log" Jun 09 01:54:22.314479 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:22.314401 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-98cf8ddb4-g7ccn_97ada88d-bc16-4fb9-98c8-0907e2ced152/nginx/0.log" Jun 09 01:54:22.423102 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:22.423078 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-sln9m_71b73ba0-84d3-43b2-ae1e-5493b500011e/manager/0.log" Jun 09 01:54:22.542994 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:22.542967 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-2w2gc_6f0cad10-da23-4ab9-9710-6cf8402e8790/manager/0.log" Jun 09 01:54:22.764253 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:22.764185 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-8jkjs_025ef825-c4e6-4e25-826b-d27d50dee348/server/0.log" Jun 09 01:54:22.889017 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:22.888987 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-tt475_9beeea4f-dbf8-4ca9-b0d5-74c15e65f365/manager/0.log" Jun 09 01:54:22.997647 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:22.997626 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c5668658c-7hfvb_2ced72e5-a97d-494c-955f-c3984f3d07fe/copy-manifests/0.log" Jun 09 01:54:23.004559 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:23.004523 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c5668658c-7hfvb_2ced72e5-a97d-494c-955f-c3984f3d07fe/manager/0.log" Jun 09 01:54:24.015293 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:24.015261 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5_08b89027-6b0f-4a9e-9707-69990447e7c0/util/0.log" Jun 09 01:54:24.022066 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:24.022044 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5_08b89027-6b0f-4a9e-9707-69990447e7c0/pull/0.log" Jun 09 01:54:24.028151 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:24.028121 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5_08b89027-6b0f-4a9e-9707-69990447e7c0/extract/0.log" Jun 09 01:54:24.140424 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:24.140401 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9_4ae6be78-17e5-4020-9462-b3c3a292dec8/pull/0.log" Jun 09 01:54:24.147327 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:24.147299 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9_4ae6be78-17e5-4020-9462-b3c3a292dec8/extract/0.log" Jun 09 01:54:24.153100 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:24.153075 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9_4ae6be78-17e5-4020-9462-b3c3a292dec8/util/0.log" Jun 09 01:54:24.266805 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:24.266737 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2_2a52df31-fe8a-447f-93a8-d1409e9ab36e/util/0.log" Jun 09 01:54:24.273219 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:24.273193 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2_2a52df31-fe8a-447f-93a8-d1409e9ab36e/pull/0.log" Jun 09 01:54:24.279238 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:24.279217 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2_2a52df31-fe8a-447f-93a8-d1409e9ab36e/extract/0.log" Jun 09 01:54:24.390708 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:24.390681 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6_5d3a828a-9d22-42a9-aa91-77290e064363/util/0.log" Jun 09 01:54:24.396481 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:24.396461 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6_5d3a828a-9d22-42a9-aa91-77290e064363/pull/0.log" Jun 09 01:54:24.402388 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:24.402365 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6_5d3a828a-9d22-42a9-aa91-77290e064363/extract/0.log" Jun 09 01:54:25.340345 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:25.340316 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-ddtm7_bb07e658-88dc-45c8-9cda-7b195aec81da/manager/0.log" Jun 09 01:54:25.796957 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:25.796925 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-vfdns_96dfe581-1560-44e0-83cb-105860269997/discovery/0.log" Jun 09 01:54:25.908438 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:25.908408 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c846984dd-5p7nh_70072497-b7f2-4a60-a5f3-29600e8953ba/kube-auth-proxy/0.log" Jun 09 01:54:26.588289 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:26.588264 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8_4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54/storage-initializer/0.log" Jun 09 01:54:26.596405 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:26.596384 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-ff5l8_4dfd0a73-9d1a-4ed6-b920-5c8898eb8d54/main/0.log" Jun 09 01:54:26.705261 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:26.705231 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-jmfhs_19fc165a-0d06-4822-a3e4-53826fd8617b/storage-initializer/0.log" Jun 09 01:54:26.712917 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:26.712889 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-jmfhs_19fc165a-0d06-4822-a3e4-53826fd8617b/main/0.log" Jun 09 01:54:26.822564 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:26.822525 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw_c72ea3e1-4e09-4e9c-b574-85d711da0d90/storage-initializer/0.log" Jun 09 01:54:26.829905 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:26.829878 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-74vsw_c72ea3e1-4e09-4e9c-b574-85d711da0d90/main/0.log" Jun 09 01:54:26.941729 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:26.941672 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f_532418cf-01a5-4897-a4b3-c51a05e5e894/storage-initializer/0.log" Jun 09 01:54:26.952070 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:26.952052 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c99b6f_532418cf-01a5-4897-a4b3-c51a05e5e894/main/0.log" Jun 09 01:54:27.061458 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:27.061427 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r_b66e01d3-cf61-40df-b048-10fdd64557f6/storage-initializer/0.log" Jun 09 01:54:27.068557 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:27.068519 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-8vf4r_b66e01d3-cf61-40df-b048-10fdd64557f6/main/0.log" Jun 09 01:54:34.616791 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:34.616738 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wgbm8_76027187-053a-4879-b929-163968a1cd82/global-pull-secret-syncer/0.log" Jun 09 01:54:34.727139 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:34.727107 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xlqwl_268a2882-9374-4669-8ed7-fefeee12a63e/konnectivity-agent/0.log" Jun 09 01:54:34.794228 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:34.794201 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-161.ec2.internal_a29d25d72d9ebc3da303cb0225e4c752/haproxy/0.log" Jun 09 01:54:37.977406 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:37.977369 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_servicemesh-operator3-57f65f65fb-67bg9_a2caae4b-a4fc-4fad-b165-dd23cb91ce11/sail-operator/0.log" Jun 09 01:54:38.493013 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:38.492973 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5_08b89027-6b0f-4a9e-9707-69990447e7c0/extract/0.log" Jun 09 01:54:38.517476 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:38.517449 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5_08b89027-6b0f-4a9e-9707-69990447e7c0/util/0.log" Jun 09 01:54:38.542167 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:38.542146 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759svfg5_08b89027-6b0f-4a9e-9707-69990447e7c0/pull/0.log" Jun 09 01:54:38.590634 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:38.590615 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9_4ae6be78-17e5-4020-9462-b3c3a292dec8/extract/0.log" Jun 09 01:54:38.617524 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:38.617499 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9_4ae6be78-17e5-4020-9462-b3c3a292dec8/util/0.log" Jun 09 01:54:38.637454 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:38.637433 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hq7r9_4ae6be78-17e5-4020-9462-b3c3a292dec8/pull/0.log" Jun 09 01:54:38.664139 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:38.664122 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2_2a52df31-fe8a-447f-93a8-d1409e9ab36e/extract/0.log" Jun 09 01:54:38.685344 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:38.685320 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2_2a52df31-fe8a-447f-93a8-d1409e9ab36e/util/0.log" Jun 09 01:54:38.707657 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:38.707619 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737djx2_2a52df31-fe8a-447f-93a8-d1409e9ab36e/pull/0.log" Jun 09 01:54:38.744089 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:38.744040 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6_5d3a828a-9d22-42a9-aa91-77290e064363/extract/0.log" Jun 09 01:54:38.780807 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:38.780784 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6_5d3a828a-9d22-42a9-aa91-77290e064363/util/0.log" Jun 09 01:54:38.801110 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:38.801087 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1l4kf6_5d3a828a-9d22-42a9-aa91-77290e064363/pull/0.log" Jun 09 01:54:39.395136 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:39.395107 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-ddtm7_bb07e658-88dc-45c8-9cda-7b195aec81da/manager/0.log" Jun 09 01:54:41.267753 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:41.267720 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ccaa03a9-10a6-4e89-bed6-206d2dd63032/alertmanager/0.log" Jun 09 01:54:41.295677 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:41.295644 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ccaa03a9-10a6-4e89-bed6-206d2dd63032/config-reloader/0.log" Jun 09 01:54:41.319591 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:41.319569 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ccaa03a9-10a6-4e89-bed6-206d2dd63032/kube-rbac-proxy-web/0.log" Jun 09 01:54:41.339879 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:41.339859 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ccaa03a9-10a6-4e89-bed6-206d2dd63032/kube-rbac-proxy/0.log" Jun 09 01:54:41.361883 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:41.361864 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ccaa03a9-10a6-4e89-bed6-206d2dd63032/kube-rbac-proxy-metric/0.log" Jun 09 01:54:41.387366 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:41.387343 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ccaa03a9-10a6-4e89-bed6-206d2dd63032/prom-label-proxy/0.log" Jun 09 01:54:41.418982 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:41.418952 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ccaa03a9-10a6-4e89-bed6-206d2dd63032/init-config-reloader/0.log" Jun 09 01:54:41.509702 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:41.509683 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-5sgdv_39d900f3-ce61-429d-a6a0-0662a6c8b6fd/kube-state-metrics/0.log" Jun 09 01:54:41.530007 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:41.529989 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-5sgdv_39d900f3-ce61-429d-a6a0-0662a6c8b6fd/kube-rbac-proxy-main/0.log" Jun 09 01:54:41.558452 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:41.558432 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-5sgdv_39d900f3-ce61-429d-a6a0-0662a6c8b6fd/kube-rbac-proxy-self/0.log" Jun 09 01:54:41.685899 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:41.685877 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4fqcv_68646a62-0c47-413d-a54c-da26b53f5903/node-exporter/0.log" Jun 09 01:54:41.705461 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:41.705442 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4fqcv_68646a62-0c47-413d-a54c-da26b53f5903/kube-rbac-proxy/0.log" Jun 09 01:54:41.725090 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:41.725073 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4fqcv_68646a62-0c47-413d-a54c-da26b53f5903/init-textfile/0.log" Jun 09 01:54:41.907720 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:41.907668 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-dp8v7_a17c6536-86cf-4097-87b7-7f7d8f864a36/kube-rbac-proxy-main/0.log" Jun 09 01:54:41.926312 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:41.926290 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-dp8v7_a17c6536-86cf-4097-87b7-7f7d8f864a36/kube-rbac-proxy-self/0.log" Jun 09 01:54:41.952187 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:41.952169 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-65f78d5c66-dp8v7_a17c6536-86cf-4097-87b7-7f7d8f864a36/openshift-state-metrics/0.log" Jun 09 01:54:42.177179 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:42.177128 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-m9ds2_e96446f7-1c13-4aec-9de2-c6ee863cfd66/prometheus-operator/0.log" Jun 09 01:54:42.202318 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:42.202296 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-m9ds2_e96446f7-1c13-4aec-9de2-c6ee863cfd66/kube-rbac-proxy/0.log" Jun 09 01:54:44.186343 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:44.186309 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d5d5bb78b-s2qt4_e247b35e-ed84-4ceb-ba24-eab0eebb8156/networking-console-plugin/0.log" Jun 09 01:54:44.822364 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:44.822334 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/2.log" Jun 09 01:54:44.831304 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:44.831282 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-4rqfp_0b11d16a-ae26-42b4-a44a-ac3b9d007579/console-operator/3.log" Jun 09 01:54:45.429327 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:45.429301 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-85cd9cc44b-zl2mh_129949c8-7227-45e4-99f7-618ee3d57270/console/0.log" Jun 09 01:54:45.468074 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:45.468047 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6657875f97-c952m_1f48718e-9823-419c-9681-5ca4a58a35d1/download-server/0.log" Jun 09 01:54:46.160673 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:46.160653 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78dd696465-shvgd_c564488d-eb77-4ed4-84a3-c3a465b41b4c/volume-data-source-validator/0.log" Jun 09 01:54:46.878918 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:46.878885 2555 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp"] Jun 09 01:54:46.879389 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:46.879373 2555 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62eb8970-32af-4bc9-ac3f-80d652a7ea78" containerName="manager" Jun 09 01:54:46.879389 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:46.879387 2555 state_mem.go:107] "Deleted CPUSet assignment" podUID="62eb8970-32af-4bc9-ac3f-80d652a7ea78" containerName="manager" Jun 09 01:54:46.879501 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:46.879488 2555 memory_manager.go:368] "RemoveStaleState removing state" podUID="62eb8970-32af-4bc9-ac3f-80d652a7ea78" containerName="manager" Jun 09 01:54:46.882981 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:46.882956 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:46.885729 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:46.885705 2555 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4cwjw\"/\"default-dockercfg-5b7d7\"" Jun 09 01:54:46.885866 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:46.885763 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4cwjw\"/\"kube-root-ca.crt\"" Jun 09 01:54:46.887703 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:46.887078 2555 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4cwjw\"/\"openshift-service-ca.crt\"" Jun 09 01:54:46.895782 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:46.895744 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp"] Jun 09 01:54:47.007130 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.007107 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fd7be30-bae9-4933-b367-f50bfb1d1992-sys\") pod \"perf-node-gather-daemonset-xjkhp\" (UID: \"4fd7be30-bae9-4933-b367-f50bfb1d1992\") " pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:47.007249 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.007144 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fd7be30-bae9-4933-b367-f50bfb1d1992-lib-modules\") pod \"perf-node-gather-daemonset-xjkhp\" (UID: \"4fd7be30-bae9-4933-b367-f50bfb1d1992\") " pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:47.007249 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.007163 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwc75\" (UniqueName: \"kubernetes.io/projected/4fd7be30-bae9-4933-b367-f50bfb1d1992-kube-api-access-mwc75\") pod \"perf-node-gather-daemonset-xjkhp\" (UID: \"4fd7be30-bae9-4933-b367-f50bfb1d1992\") " pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:47.007249 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.007227 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fd7be30-bae9-4933-b367-f50bfb1d1992-proc\") pod \"perf-node-gather-daemonset-xjkhp\" (UID: \"4fd7be30-bae9-4933-b367-f50bfb1d1992\") " pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:47.007352 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.007299 2555 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fd7be30-bae9-4933-b367-f50bfb1d1992-podres\") pod \"perf-node-gather-daemonset-xjkhp\" (UID: \"4fd7be30-bae9-4933-b367-f50bfb1d1992\") " pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:47.108663 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.108641 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fd7be30-bae9-4933-b367-f50bfb1d1992-podres\") pod \"perf-node-gather-daemonset-xjkhp\" (UID: \"4fd7be30-bae9-4933-b367-f50bfb1d1992\") " pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:47.108786 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.108689 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fd7be30-bae9-4933-b367-f50bfb1d1992-sys\") pod \"perf-node-gather-daemonset-xjkhp\" (UID: \"4fd7be30-bae9-4933-b367-f50bfb1d1992\") " pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:47.108786 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.108729 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fd7be30-bae9-4933-b367-f50bfb1d1992-lib-modules\") pod \"perf-node-gather-daemonset-xjkhp\" (UID: \"4fd7be30-bae9-4933-b367-f50bfb1d1992\") " pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:47.108786 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.108770 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwc75\" (UniqueName: \"kubernetes.io/projected/4fd7be30-bae9-4933-b367-f50bfb1d1992-kube-api-access-mwc75\") pod \"perf-node-gather-daemonset-xjkhp\" (UID: \"4fd7be30-bae9-4933-b367-f50bfb1d1992\") " pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:47.108918 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.108795 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fd7be30-bae9-4933-b367-f50bfb1d1992-sys\") pod \"perf-node-gather-daemonset-xjkhp\" (UID: \"4fd7be30-bae9-4933-b367-f50bfb1d1992\") " pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:47.108918 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.108804 2555 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fd7be30-bae9-4933-b367-f50bfb1d1992-proc\") pod \"perf-node-gather-daemonset-xjkhp\" (UID: \"4fd7be30-bae9-4933-b367-f50bfb1d1992\") " pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:47.108918 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.108836 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fd7be30-bae9-4933-b367-f50bfb1d1992-podres\") pod \"perf-node-gather-daemonset-xjkhp\" (UID: \"4fd7be30-bae9-4933-b367-f50bfb1d1992\") " pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:47.108918 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.108845 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fd7be30-bae9-4933-b367-f50bfb1d1992-proc\") pod \"perf-node-gather-daemonset-xjkhp\" (UID: \"4fd7be30-bae9-4933-b367-f50bfb1d1992\") " pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:47.108918 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.108852 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fd7be30-bae9-4933-b367-f50bfb1d1992-lib-modules\") pod \"perf-node-gather-daemonset-xjkhp\" (UID: \"4fd7be30-bae9-4933-b367-f50bfb1d1992\") " pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:47.116457 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.116436 2555 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwc75\" (UniqueName: \"kubernetes.io/projected/4fd7be30-bae9-4933-b367-f50bfb1d1992-kube-api-access-mwc75\") pod \"perf-node-gather-daemonset-xjkhp\" (UID: \"4fd7be30-bae9-4933-b367-f50bfb1d1992\") " pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:47.199420 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.199370 2555 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:47.440453 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.440425 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5vfr5_c4cca482-39bd-4a22-bed7-e85a83b32b47/dns/0.log" Jun 09 01:54:47.460595 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.460506 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5vfr5_c4cca482-39bd-4a22-bed7-e85a83b32b47/kube-rbac-proxy/0.log" Jun 09 01:54:47.537376 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.537351 2555 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp"] Jun 09 01:54:47.537909 ip-10-0-143-161 kubenswrapper[2555]: W0609 01:54:47.537884 2555 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4fd7be30_bae9_4933_b367_f50bfb1d1992.slice/crio-842fca4481f91864896d889549a1ca564f850571e171bcdafffaf420c8994400 WatchSource:0}: Error finding container 842fca4481f91864896d889549a1ca564f850571e171bcdafffaf420c8994400: Status 404 returned error can't find the container with id 842fca4481f91864896d889549a1ca564f850571e171bcdafffaf420c8994400 Jun 09 01:54:47.617695 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:47.617667 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rs95s_8e1cf77d-cef0-4363-a02f-f0887da4e984/dns-node-resolver/0.log" Jun 09 01:54:48.121725 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:48.121699 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5f9bcfd4d8-m4rzd_e52b9d7c-548a-4cf5-933c-7e2e0969e797/registry/0.log" Jun 09 01:54:48.138620 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:48.138561 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5f9bcfd4d8-m4rzd_e52b9d7c-548a-4cf5-933c-7e2e0969e797/registry/1.log" Jun 09 01:54:48.183453 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:48.183423 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-87mlr_4108bdff-ba06-42ab-9b64-a7be1cf9c8a4/node-ca/0.log" Jun 09 01:54:48.377047 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:48.376984 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" event={"ID":"4fd7be30-bae9-4933-b367-f50bfb1d1992","Type":"ContainerStarted","Data":"beef6626cec8c0d37375c28e8b75081e4d9cba02df705843037fc09767ec5853"} Jun 09 01:54:48.377047 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:48.377032 2555 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" event={"ID":"4fd7be30-bae9-4933-b367-f50bfb1d1992","Type":"ContainerStarted","Data":"842fca4481f91864896d889549a1ca564f850571e171bcdafffaf420c8994400"} Jun 09 01:54:48.377206 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:48.377141 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:48.394667 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:48.394626 2555 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" podStartSLOduration=2.39461107 podStartE2EDuration="2.39461107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 01:54:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 01:54:48.392443505 +0000 UTC m=+2349.410747720" watchObservedRunningTime="2026-06-09 01:54:48.39461107 +0000 UTC m=+2349.412915294" Jun 09 01:54:49.232075 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:49.232049 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-vfdns_96dfe581-1560-44e0-83cb-105860269997/discovery/0.log" Jun 09 01:54:49.251685 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:49.251666 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c846984dd-5p7nh_70072497-b7f2-4a60-a5f3-29600e8953ba/kube-auth-proxy/0.log" Jun 09 01:54:49.913315 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:49.913287 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-prpjp_5e0bcec3-73e6-4a2a-bc46-2848c31d9ce9/serve-healthcheck-canary/0.log" Jun 09 01:54:50.485530 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:50.485500 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-896549976-jn4jq_b9464d89-5eba-4239-aff5-5f2430ecd991/insights-operator/0.log" Jun 09 01:54:50.488227 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:50.488201 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-896549976-jn4jq_b9464d89-5eba-4239-aff5-5f2430ecd991/insights-operator/1.log" Jun 09 01:54:50.634088 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:50.634054 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qwn6v_b96de010-76d0-4a70-9437-d2d54dfa7631/kube-rbac-proxy/0.log" Jun 09 01:54:50.653223 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:50.653205 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qwn6v_b96de010-76d0-4a70-9437-d2d54dfa7631/exporter/0.log" Jun 09 01:54:50.673181 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:50.673159 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qwn6v_b96de010-76d0-4a70-9437-d2d54dfa7631/extractor/0.log" Jun 09 01:54:52.838903 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:52.838871 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-98cf8ddb4-g7ccn_97ada88d-bc16-4fb9-98c8-0907e2ced152/nginx/0.log" Jun 09 01:54:52.862683 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:52.862652 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-sln9m_71b73ba0-84d3-43b2-ae1e-5493b500011e/manager/0.log" Jun 09 01:54:53.000788 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:53.000763 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-2w2gc_6f0cad10-da23-4ab9-9710-6cf8402e8790/manager/0.log" Jun 09 01:54:53.108433 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:53.108370 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-8jkjs_025ef825-c4e6-4e25-826b-d27d50dee348/server/0.log" Jun 09 01:54:53.180669 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:53.180648 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-tt475_9beeea4f-dbf8-4ca9-b0d5-74c15e65f365/manager/0.log" Jun 09 01:54:53.202226 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:53.202203 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c5668658c-7hfvb_2ced72e5-a97d-494c-955f-c3984f3d07fe/manager/0.log" Jun 09 01:54:53.225866 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:53.225848 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c5668658c-7hfvb_2ced72e5-a97d-494c-955f-c3984f3d07fe/copy-manifests/0.log" Jun 09 01:54:54.394063 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:54.394039 2555 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4cwjw/perf-node-gather-daemonset-xjkhp" Jun 09 01:54:54.829229 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:54.829205 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-2dh9t_e772b6e0-56bd-48ef-ac3e-00b39d17c61c/openshift-lws-operator/0.log" Jun 09 01:54:59.858176 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:59.858138 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-vfdns_96dfe581-1560-44e0-83cb-105860269997/discovery/0.log" Jun 09 01:54:59.870151 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:54:59.870119 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c846984dd-5p7nh_70072497-b7f2-4a60-a5f3-29600e8953ba/kube-auth-proxy/0.log" Jun 09 01:55:00.797079 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:55:00.797035 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55bb7c4d49-f9xcx_0855fbf3-db84-4b7c-b61a-c5dc9a63e530/kube-storage-version-migrator-operator/1.log" Jun 09 01:55:00.798739 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:55:00.798713 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55bb7c4d49-f9xcx_0855fbf3-db84-4b7c-b61a-c5dc9a63e530/kube-storage-version-migrator-operator/0.log" Jun 09 01:55:01.145538 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:55:01.145458 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-vfdns_96dfe581-1560-44e0-83cb-105860269997/discovery/0.log" Jun 09 01:55:01.160700 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:55:01.160668 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c846984dd-5p7nh_70072497-b7f2-4a60-a5f3-29600e8953ba/kube-auth-proxy/0.log" Jun 09 01:55:02.109154 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:55:02.109119 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zrlt_fbd5c78f-a01c-41d6-81ce-fd01b8b78e43/kube-multus-additional-cni-plugins/0.log" Jun 09 01:55:02.131538 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:55:02.131503 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zrlt_fbd5c78f-a01c-41d6-81ce-fd01b8b78e43/egress-router-binary-copy/0.log" Jun 09 01:55:02.153898 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:55:02.153874 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zrlt_fbd5c78f-a01c-41d6-81ce-fd01b8b78e43/cni-plugins/0.log" Jun 09 01:55:02.177826 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:55:02.177793 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zrlt_fbd5c78f-a01c-41d6-81ce-fd01b8b78e43/bond-cni-plugin/0.log" Jun 09 01:55:02.205254 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:55:02.205235 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zrlt_fbd5c78f-a01c-41d6-81ce-fd01b8b78e43/routeoverride-cni/0.log" Jun 09 01:55:02.231816 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:55:02.231797 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zrlt_fbd5c78f-a01c-41d6-81ce-fd01b8b78e43/whereabouts-cni-bincopy/0.log" Jun 09 01:55:02.255257 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:55:02.255235 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5zrlt_fbd5c78f-a01c-41d6-81ce-fd01b8b78e43/whereabouts-cni/0.log" Jun 09 01:55:02.500582 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:55:02.500497 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-sl99s_05fcb8ca-c9b6-46ed-82f5-cdfed42a61d3/kube-multus/0.log" Jun 09 01:55:02.605962 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:55:02.605929 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8prh2_8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19/network-metrics-daemon/0.log" Jun 09 01:55:02.623993 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:55:02.623972 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8prh2_8ad5e6dd-1c8b-4c90-a4b6-df145fcbbf19/kube-rbac-proxy/0.log" Jun 09 01:55:02.931247 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:55:02.931224 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-vfdns_96dfe581-1560-44e0-83cb-105860269997/discovery/0.log" Jun 09 01:55:02.941664 ip-10-0-143-161 kubenswrapper[2555]: I0609 01:55:02.941646 2555 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c846984dd-5p7nh_70072497-b7f2-4a60-a5f3-29600e8953ba/kube-auth-proxy/0.log"