Jun 11 16:08:44.061249 ip-10-0-140-111 systemd[1]: Starting Kubernetes Kubelet... Jun 11 16:08:44.405687 ip-10-0-140-111 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 16:08:44.405687 ip-10-0-140-111 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 11 16:08:44.405687 ip-10-0-140-111 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 16:08:44.405687 ip-10-0-140-111 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 11 16:08:44.405687 ip-10-0-140-111 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 11 16:08:44.406662 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.406517 2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 11 16:08:44.411903 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411885 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:08:44.411903 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411902 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411906 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411910 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411913 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411916 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411936 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411939 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411942 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411944 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411949 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411954 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411957 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411961 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411963 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411966 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411969 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411971 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411974 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411976 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:08:44.411988 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411979 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411981 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411984 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411986 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411990 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411993 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411996 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.411999 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412001 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412004 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412007 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412009 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412012 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412014 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412017 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412019 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412022 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412024 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412027 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412029 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:08:44.412477 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412031 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412034 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412037 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412040 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412042 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412044 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412049 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412052 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412055 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412058 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412061 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412063 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412066 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412069 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412072 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412074 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412077 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412081 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412084 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:08:44.412972 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412087 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412090 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412092 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412095 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412097 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412099 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412102 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412105 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412107 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412111 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412113 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412116 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412118 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412121 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412123 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412126 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412129 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412132 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412134 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412138 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:08:44.413439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412141 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412144 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412146 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412149 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412151 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412154 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412156 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412160 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412162 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412165 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412167 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412170 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412172 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412175 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412177 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412179 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412182 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412184 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412187 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.412189 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:08:44.413940 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413479 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413486 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413490 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413494 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413498 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413501 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413504 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413507 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413509 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413514 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413517 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413520 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413523 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413525 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413528 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413531 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413534 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413538 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413541 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:08:44.414414 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413544 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413547 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413549 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413552 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413554 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413557 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413559 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413562 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413564 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413567 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413570 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413572 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413575 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413577 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413580 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413582 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413585 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413587 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413590 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413592 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:08:44.414890 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413595 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413597 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413600 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413603 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413606 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413608 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413611 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413613 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413616 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413618 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413621 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413624 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413627 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413629 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413632 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413635 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413637 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413640 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413643 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413645 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:08:44.415406 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413647 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413650 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413652 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413655 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413657 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413659 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413662 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413664 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413666 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413669 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413671 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413674 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413676 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413679 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413681 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413684 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413687 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413690 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413693 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:08:44.415976 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413696 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413699 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413701 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413704 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413706 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413709 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413712 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413714 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413717 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413720 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413724 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413727 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413730 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413732 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413735 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413737 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413739 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413742 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413744 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413747 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:08:44.416454 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.413750 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414157 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414164 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414167 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414169 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414173 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414176 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414178 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414181 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414184 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414187 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414190 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414195 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414197 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414200 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414205 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414207 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414210 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414213 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414215 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:08:44.416953 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414218 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414221 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414223 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414227 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414230 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414233 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414236 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414239 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414241 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414244 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414247 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414249 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414252 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414255 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414257 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414260 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414262 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414265 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414267 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:08:44.417484 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414270 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414272 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414275 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414280 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414283 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414287 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414290 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414293 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414297 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414299 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414302 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414304 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414307 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414309 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414312 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414316 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414319 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414322 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414324 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:08:44.417960 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414327 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414330 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414332 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414334 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414337 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414339 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414342 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414344 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414347 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414349 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414352 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414354 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414357 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414359 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414362 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414364 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414367 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414370 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414373 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414376 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:08:44.418421 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414378 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414382 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414385 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414387 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414390 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414392 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414395 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414397 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414400 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414402 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414405 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414408 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414410 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414413 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414417 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414420 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414423 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414426 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414429 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414431 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:08:44.419005 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414434 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.414436 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414509 2573 flags.go:64] FLAG: --address="0.0.0.0" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414520 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414528 2573 flags.go:64] FLAG: --anonymous-auth="true" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414532 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414536 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414539 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414543 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414547 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414552 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414555 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414558 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414563 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414566 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414569 2573 flags.go:64] FLAG: --cgroup-root="" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414572 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414575 2573 flags.go:64] FLAG: --client-ca-file="" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414577 2573 flags.go:64] FLAG: --cloud-provider="external" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414580 2573 flags.go:64] FLAG: --cluster-dns="[]" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414584 2573 flags.go:64] FLAG: --cluster-domain="" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414587 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414590 2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414593 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 11 16:08:44.419521 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414596 2573 flags.go:64] FLAG: --container-log-max-files="5" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414599 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414602 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414605 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414609 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414612 2573 flags.go:64] FLAG: --contention-profiling="false" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414614 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414617 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414620 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414623 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414627 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414629 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414632 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414635 2573 flags.go:64] FLAG: --enable-load-reader="false" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414638 2573 flags.go:64] FLAG: --enable-server="true" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414641 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414645 2573 flags.go:64] FLAG: --event-burst="100" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414648 2573 flags.go:64] FLAG: --event-qps="50" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414652 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414655 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414658 2573 flags.go:64] FLAG: --eviction-hard="" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414663 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414666 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414669 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414672 2573 flags.go:64] FLAG: --eviction-soft="" Jun 11 16:08:44.420161 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414675 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414677 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414680 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414683 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414686 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414689 2573 flags.go:64] FLAG: --fail-swap-on="true" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414691 2573 flags.go:64] FLAG: --feature-gates="" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414695 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414697 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414700 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414703 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414706 2573 flags.go:64] FLAG: --healthz-port="10248" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414709 2573 flags.go:64] FLAG: --help="false" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414712 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-140-111.ec2.internal" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414715 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414718 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414721 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414724 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414727 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414730 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414733 2573 flags.go:64] FLAG: --image-service-endpoint="" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414735 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414738 2573 flags.go:64] FLAG: --kube-api-burst="100" Jun 11 16:08:44.420752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414741 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414743 2573 flags.go:64] FLAG: --kube-api-qps="50" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414746 2573 flags.go:64] FLAG: --kube-reserved="" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414749 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414752 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414757 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414759 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414762 2573 flags.go:64] FLAG: --lock-file="" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414765 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414768 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414771 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414776 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414779 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414782 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414785 2573 flags.go:64] FLAG: --logging-format="text" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414788 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414791 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414793 2573 flags.go:64] FLAG: --manifest-url="" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414796 2573 flags.go:64] FLAG: --manifest-url-header="" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414800 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414803 2573 flags.go:64] FLAG: --max-open-files="1000000" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414807 2573 flags.go:64] FLAG: --max-pods="110" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414810 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414813 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414815 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jun 11 16:08:44.421394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414818 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414821 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414824 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414826 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414833 2573 flags.go:64] FLAG: --node-status-max-images="50" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414836 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414839 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414842 2573 flags.go:64] FLAG: --pod-cidr="" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414845 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414850 2573 flags.go:64] FLAG: --pod-manifest-path="" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414853 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414856 2573 flags.go:64] FLAG: --pods-per-core="0" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414860 2573 flags.go:64] FLAG: --port="10250" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414864 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414867 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-076015148a52e0030" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414870 2573 flags.go:64] FLAG: --qos-reserved="" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414873 2573 flags.go:64] FLAG: --read-only-port="10255" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414876 2573 flags.go:64] FLAG: --register-node="true" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414879 2573 flags.go:64] FLAG: --register-with-taints="" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414882 2573 flags.go:64] FLAG: --registry-burst="10" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414885 2573 flags.go:64] FLAG: --registry-qps="5" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414887 2573 flags.go:64] FLAG: --reserved-cpus="" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414890 2573 flags.go:64] FLAG: --reserved-memory="" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414894 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414897 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 11 16:08:44.422036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414900 2573 flags.go:64] FLAG: --rotate-certificates="false" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414902 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414905 2573 flags.go:64] FLAG: --runonce="false" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414908 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414911 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414914 2573 flags.go:64] FLAG: --seccomp-default="false" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414916 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414938 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414944 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414948 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414952 2573 flags.go:64] FLAG: --storage-driver-password="root" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414954 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414957 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414960 2573 flags.go:64] FLAG: --storage-driver-user="root" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414962 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414965 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414969 2573 flags.go:64] FLAG: --system-cgroups="" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414972 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414977 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414980 2573 flags.go:64] FLAG: --tls-cert-file="" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414993 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.414998 2573 flags.go:64] FLAG: --tls-min-version="" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.415001 2573 flags.go:64] FLAG: --tls-private-key-file="" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.415003 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.415006 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 11 16:08:44.422666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.415009 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.415012 2573 flags.go:64] FLAG: --v="2" Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.415016 2573 flags.go:64] FLAG: --version="false" Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.415021 2573 flags.go:64] FLAG: --vmodule="" Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.415024 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.415027 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415144 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415148 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415151 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415154 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415156 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415159 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415162 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415164 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415167 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415171 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415174 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415176 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415179 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415181 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415184 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:08:44.423297 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415186 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415189 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415191 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415194 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415196 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415199 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415202 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415206 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415208 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415211 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415213 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415216 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415220 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415223 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415227 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415231 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415234 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415237 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415239 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:08:44.423812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415242 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415245 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415248 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415250 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415252 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415255 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415258 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415262 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415264 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415267 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415269 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415272 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415274 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415277 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415279 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415282 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415284 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415287 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415289 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415294 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:08:44.424356 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415296 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415299 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415302 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415304 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415307 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415310 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415312 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415315 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415317 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415320 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415322 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415324 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415327 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415329 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415332 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415334 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415337 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415339 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415342 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415346 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:08:44.424826 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415348 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415351 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415353 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415356 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415360 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415363 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415366 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415368 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415371 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415374 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415377 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415381 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415384 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415386 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415390 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415392 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415395 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415398 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415400 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:08:44.425339 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415403 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:08:44.425843 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415405 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:08:44.425843 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415408 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:08:44.425843 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415410 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:08:44.425843 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415413 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:08:44.425843 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.415416 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:08:44.425843 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.415971 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 16:08:44.425843 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.422571 2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 11 16:08:44.425843 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.422587 2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 11 16:08:44.425843 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.422605 2573 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 11 16:08:44.425843 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.422611 2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 11 16:08:44.425843 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422666 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:08:44.425843 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422673 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:08:44.425843 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422676 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:08:44.425843 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422680 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:08:44.425843 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422683 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422686 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422689 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422692 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422694 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422697 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422699 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422702 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422704 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422707 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422710 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422712 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422716 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422718 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422721 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422723 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422726 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422728 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422731 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422735 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:08:44.426272 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422738 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422741 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422744 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422746 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422749 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422752 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422755 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422757 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422760 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422762 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422765 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422768 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422770 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422772 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422775 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422778 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422781 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422783 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422785 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422788 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:08:44.426764 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422791 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422793 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422796 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422798 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422801 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422804 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422806 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422809 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422811 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422814 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422816 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422818 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422821 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422823 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422827 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422831 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422833 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422837 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422840 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:08:44.427288 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422842 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422846 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422849 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422852 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422855 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422858 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422861 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422863 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422866 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422869 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422872 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422875 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422877 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422880 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422883 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422886 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422888 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422891 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422894 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:08:44.427747 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422897 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:08:44.428240 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422899 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:08:44.428240 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422902 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:08:44.428240 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422904 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:08:44.428240 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422907 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:08:44.428240 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422909 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:08:44.428240 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422912 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:08:44.428240 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422914 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:08:44.428240 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422916 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:08:44.428240 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422936 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:08:44.428240 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422939 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:08:44.428240 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422943 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:08:44.428240 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422945 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:08:44.428240 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422948 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:08:44.428240 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422950 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:08:44.428240 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422953 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:08:44.428240 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.422955 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:08:44.428632 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.422961 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 16:08:44.428632 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423102 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 11 16:08:44.428632 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423108 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 11 16:08:44.428632 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423111 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 11 16:08:44.428632 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423114 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 11 16:08:44.428632 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423117 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 11 16:08:44.428632 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423120 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 11 16:08:44.428632 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423122 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 11 16:08:44.428632 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423125 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 11 16:08:44.428632 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423128 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 11 16:08:44.428632 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423130 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 11 16:08:44.428632 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423133 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 11 16:08:44.428632 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423136 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 11 16:08:44.428632 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423138 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 11 16:08:44.428632 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423140 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423143 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423146 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423149 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423151 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423154 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423156 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423159 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423161 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423164 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423166 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423168 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423171 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423174 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423176 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423179 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423181 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423184 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423186 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423189 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 11 16:08:44.429008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423191 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423194 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423196 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423199 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423201 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423203 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423206 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423208 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423211 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423215 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423219 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423221 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423224 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423226 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423229 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423232 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423234 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423237 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423239 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 11 16:08:44.429518 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423241 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423244 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423246 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423250 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423253 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423256 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423259 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423261 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423264 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423266 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423269 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423271 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423274 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423276 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423279 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423281 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423285 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423288 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423290 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 11 16:08:44.429997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423293 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423296 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423299 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423302 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423304 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423307 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423310 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423313 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423316 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423318 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423327 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423330 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423333 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423335 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423337 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423340 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423342 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423345 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423348 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423350 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 11 16:08:44.430468 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423353 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 11 16:08:44.431317 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423355 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 11 16:08:44.431317 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423358 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 11 16:08:44.431317 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423360 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 11 16:08:44.431317 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423363 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 11 16:08:44.431317 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423365 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 11 16:08:44.431317 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423368 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 11 16:08:44.431317 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:44.423370 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 11 16:08:44.431317 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.423375 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 11 16:08:44.431317 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.423483 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jun 11 16:08:44.431317 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.426176 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 11 16:08:44.431317 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.427348 2573 server.go:1019] "Starting client certificate rotation" Jun 11 16:08:44.431317 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.427452 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 16:08:44.431317 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.427492 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 11 16:08:44.431834 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.429530 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 16:08:44.432976 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.432947 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 11 16:08:44.444237 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.444217 2573 log.go:25] "Validated CRI v1 runtime API" Jun 11 16:08:44.446797 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.446782 2573 log.go:25] "Validated CRI v1 image API" Jun 11 16:08:44.447654 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.447635 2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 11 16:08:44.450009 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.449984 2573 fs.go:135] Filesystem UUIDs: map[3bf5423c-502f-40bf-8ee4-e87984249d3f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f9bae055-46d6-4f61-b20f-82520e94be59:/dev/nvme0n1p3] Jun 11 16:08:44.450056 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.450010 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 11 16:08:44.454397 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.454287 2573 manager.go:216] Machine: {Timestamp:2026-06-11 16:08:44.453770886 +0000 UTC m=+0.287883416 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101411 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24cf3503e713d829cb06f1d3ac68d4 SystemUUID:ec24cf35-03e7-13d8-29cb-06f1d3ac68d4 BootID:d2cd1182-bbb7-4214-8b92-c24db7ae1c5b Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:19:1b:e7:b3:8f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:19:1b:e7:b3:8f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:69:84:5a:7d:22 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 11 16:08:44.454397 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.454391 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 11 16:08:44.454497 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.454468 2573 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 11 16:08:44.457079 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.457052 2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 11 16:08:44.457223 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.457082 2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-111.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 11 16:08:44.457770 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.457761 2573 topology_manager.go:138] "Creating topology manager with none policy" Jun 11 16:08:44.457801 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.457773 2573 container_manager_linux.go:309] "Creating device plugin manager" Jun 11 16:08:44.457801 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.457782 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 16:08:44.458438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.458423 2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 11 16:08:44.459344 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.459332 2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 11 16:08:44.463317 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.463304 2573 state_mem.go:36] "Initialized new in-memory state store" Jun 11 16:08:44.463439 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.463429 2573 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 11 16:08:44.463480 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.463471 2573 kubelet.go:486] "Attempting to sync node with API server" Jun 11 16:08:44.463512 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.463488 2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 11 16:08:44.463539 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.463520 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 11 16:08:44.463539 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.463533 2573 kubelet.go:398] "Adding apiserver pod source" Jun 11 16:08:44.463608 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.463548 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 11 16:08:44.463991 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.463979 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 16:08:44.464027 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.464004 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 11 16:08:44.471411 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.471357 2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 11 16:08:44.471592 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.471571 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 16:08:44.473072 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.473057 2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 11 16:08:44.473131 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.473090 2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 11 16:08:44.474562 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.474549 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 11 16:08:44.474595 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.474568 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 11 16:08:44.474595 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.474574 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 11 16:08:44.474595 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.474581 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 11 16:08:44.474595 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.474590 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 11 16:08:44.474595 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.474596 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 11 16:08:44.474730 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.474605 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 11 16:08:44.474730 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.474610 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 11 16:08:44.474730 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.474621 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 11 16:08:44.474730 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.474626 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 11 16:08:44.474730 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.474635 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 11 16:08:44.474730 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.474640 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 11 16:08:44.475310 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.475297 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 11 16:08:44.475352 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.475314 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 11 16:08:44.477813 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.477782 2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-111.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 11 16:08:44.477813 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.477795 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 11 16:08:44.477813 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.477781 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-111.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 11 16:08:44.482237 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.482218 2573 server.go:1268] "Started kubelet" Jun 11 16:08:44.482309 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.482286 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 11 16:08:44.482548 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.482395 2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 11 16:08:44.482701 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.482684 2573 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 11 16:08:44.485064 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.485025 2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 11 16:08:44.485180 ip-10-0-140-111 systemd[1]: Started Kubernetes Kubelet. Jun 11 16:08:44.486705 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.486686 2573 server.go:310] "Adding debug handlers to kubelet server" Jun 11 16:08:44.489179 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.488344 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-111.ec2.internal.18b8129e79c6136b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-111.ec2.internal,UID:ip-10-0-140-111.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-111.ec2.internal,},FirstTimestamp:2026-06-11 16:08:44.482188139 +0000 UTC m=+0.316300669,LastTimestamp:2026-06-11 16:08:44.482188139 +0000 UTC m=+0.316300669,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-111.ec2.internal,}" Jun 11 16:08:44.491018 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.490995 2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 11 16:08:44.491285 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.491266 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 11 16:08:44.491285 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.491280 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 11 16:08:44.491421 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.491332 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-111.ec2.internal\" not found" Jun 11 16:08:44.491935 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.491902 2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 11 16:08:44.491935 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.491915 2573 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 11 16:08:44.492086 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.491949 2573 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 11 16:08:44.492086 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.492019 2573 factory.go:55] Registering systemd factory Jun 11 16:08:44.492086 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.492036 2573 factory.go:223] Registration of the systemd container factory successfully Jun 11 16:08:44.492220 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.492090 2573 reconstruct.go:97] "Volume reconstruction finished" Jun 11 16:08:44.492220 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.492099 2573 reconciler.go:29] "Reconciler: start to sync state" Jun 11 16:08:44.492329 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.492274 2573 factory.go:153] Registering CRI-O factory Jun 11 16:08:44.492329 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.492288 2573 factory.go:223] Registration of the crio container factory successfully Jun 11 16:08:44.492427 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.492338 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 11 16:08:44.492427 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.492363 2573 factory.go:103] Registering Raw factory Jun 11 16:08:44.492427 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.492378 2573 manager.go:1196] Started watching for new ooms in manager Jun 11 16:08:44.492784 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.492771 2573 manager.go:349] Starting recovery of all containers Jun 11 16:08:44.503122 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.503102 2573 manager.go:354] Recovery completed Jun 11 16:08:44.504657 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.504633 2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-111.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 11 16:08:44.504932 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.504896 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 11 16:08:44.508035 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.508019 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:08:44.509825 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.509809 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:08:44.509895 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.509838 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:08:44.509895 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.509849 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:08:44.510323 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.510309 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 11 16:08:44.510364 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.510321 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:08:44.510396 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.510324 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 11 16:08:44.510432 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.510404 2573 state_mem.go:36] "Initialized new in-memory state store" Jun 11 16:08:44.510908 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.510890 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:08:44.511016 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.510931 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:08:44.511016 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.510945 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:08:44.512638 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.512624 2573 policy_none.go:49] "None policy: Start" Jun 11 16:08:44.512712 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.512642 2573 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 11 16:08:44.512712 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.512652 2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 11 16:08:44.513189 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.513110 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-111.ec2.internal.18b8129e7b6bc83c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-111.ec2.internal,UID:ip-10-0-140-111.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-111.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-111.ec2.internal,},FirstTimestamp:2026-06-11 16:08:44.509825084 +0000 UTC m=+0.343937617,LastTimestamp:2026-06-11 16:08:44.509825084 +0000 UTC m=+0.343937617,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-111.ec2.internal,}" Jun 11 16:08:44.514710 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.514695 2573 policy_none.go:47] "Start" Jun 11 16:08:44.524067 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.524002 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-111.ec2.internal.18b8129e7b6c0e14 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-111.ec2.internal,UID:ip-10-0-140-111.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-111.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-111.ec2.internal,},FirstTimestamp:2026-06-11 16:08:44.509842964 +0000 UTC m=+0.343955493,LastTimestamp:2026-06-11 16:08:44.509842964 +0000 UTC m=+0.343955493,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-111.ec2.internal,}" Jun 11 16:08:44.535123 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.535030 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-111.ec2.internal.18b8129e7b6c364e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-111.ec2.internal,UID:ip-10-0-140-111.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-140-111.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-140-111.ec2.internal,},FirstTimestamp:2026-06-11 16:08:44.509853262 +0000 UTC m=+0.343965793,LastTimestamp:2026-06-11 16:08:44.509853262 +0000 UTC m=+0.343965793,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-111.ec2.internal,}" Jun 11 16:08:44.554042 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.553964 2573 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-111.ec2.internal.18b8129e7b6bc83c\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-111.ec2.internal.18b8129e7b6bc83c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-111.ec2.internal,UID:ip-10-0-140-111.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-111.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-111.ec2.internal,},FirstTimestamp:2026-06-11 16:08:44.509825084 +0000 UTC m=+0.343937617,LastTimestamp:2026-06-11 16:08:44.510905117 +0000 UTC m=+0.345017647,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-111.ec2.internal,}" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.556586 2573 manager.go:337] "Starting Device Plugin manager" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.556630 2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.556640 2573 server.go:87] "Starting device plugin registration server" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.556941 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.556961 2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.557053 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.557160 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.557170 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.557937 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kdnzr" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.558079 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.558117 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-111.ec2.internal\" not found" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.560585 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.561966 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.562000 2573 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.562018 2573 kubelet.go:2448] "Starting kubelet main sync loop" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.562047 2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.562620 2573 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-140-111.ec2.internal.18b8129e7b6c0e14\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-111.ec2.internal.18b8129e7b6c0e14 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-111.ec2.internal,UID:ip-10-0-140-111.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-140-111.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-140-111.ec2.internal,},FirstTimestamp:2026-06-11 16:08:44.509842964 +0000 UTC m=+0.343955493,LastTimestamp:2026-06-11 16:08:44.510939461 +0000 UTC m=+0.345051994,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-111.ec2.internal,}" Jun 11 16:08:44.571618 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.568105 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kdnzr" Jun 11 16:08:44.574905 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.574887 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 16:08:44.657256 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.657221 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:08:44.658427 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.658410 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:08:44.658511 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.658440 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:08:44.658511 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.658450 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:08:44.658511 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.658477 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-111.ec2.internal" Jun 11 16:08:44.662548 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.662530 2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-111.ec2.internal"] Jun 11 16:08:44.662619 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.662590 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:08:44.663884 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.663869 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:08:44.663979 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.663894 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:08:44.663979 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.663903 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:08:44.666188 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.666176 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:08:44.666310 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.666294 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal" Jun 11 16:08:44.666388 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.666326 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:08:44.666951 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.666916 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:08:44.667022 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.666966 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:08:44.667022 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.666978 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:08:44.667092 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.666937 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:08:44.667092 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.667047 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:08:44.667092 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.667060 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:08:44.667197 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.667168 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-111.ec2.internal" Jun 11 16:08:44.667197 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.667184 2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-111.ec2.internal\": node \"ip-10-0-140-111.ec2.internal\" not found" Jun 11 16:08:44.669332 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.669318 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-111.ec2.internal" Jun 11 16:08:44.669405 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.669341 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 11 16:08:44.669983 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.669967 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasSufficientMemory" Jun 11 16:08:44.670069 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.669988 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasNoDiskPressure" Jun 11 16:08:44.670069 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.669998 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeHasSufficientPID" Jun 11 16:08:44.693821 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.693791 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3023c529b9a574aede626b35cf1e00da-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal\" (UID: \"3023c529b9a574aede626b35cf1e00da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal" Jun 11 16:08:44.693976 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.693823 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3023c529b9a574aede626b35cf1e00da-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal\" (UID: \"3023c529b9a574aede626b35cf1e00da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal" Jun 11 16:08:44.693976 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.693847 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3154ce17f76996f6fd8c8d37ce0304d4-config\") pod \"kube-apiserver-proxy-ip-10-0-140-111.ec2.internal\" (UID: \"3154ce17f76996f6fd8c8d37ce0304d4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-111.ec2.internal" Jun 11 16:08:44.698757 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.698738 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-111.ec2.internal\" not found" node="ip-10-0-140-111.ec2.internal" Jun 11 16:08:44.703129 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.703112 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-111.ec2.internal\" not found" node="ip-10-0-140-111.ec2.internal" Jun 11 16:08:44.713891 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.713874 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-111.ec2.internal\" not found" Jun 11 16:08:44.794462 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.794441 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3023c529b9a574aede626b35cf1e00da-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal\" (UID: \"3023c529b9a574aede626b35cf1e00da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal" Jun 11 16:08:44.794574 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.794474 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3023c529b9a574aede626b35cf1e00da-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal\" (UID: \"3023c529b9a574aede626b35cf1e00da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal" Jun 11 16:08:44.794574 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.794500 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3154ce17f76996f6fd8c8d37ce0304d4-config\") pod \"kube-apiserver-proxy-ip-10-0-140-111.ec2.internal\" (UID: \"3154ce17f76996f6fd8c8d37ce0304d4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-111.ec2.internal" Jun 11 16:08:44.794574 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.794549 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3023c529b9a574aede626b35cf1e00da-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal\" (UID: \"3023c529b9a574aede626b35cf1e00da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal" Jun 11 16:08:44.794676 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.794549 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3154ce17f76996f6fd8c8d37ce0304d4-config\") pod \"kube-apiserver-proxy-ip-10-0-140-111.ec2.internal\" (UID: \"3154ce17f76996f6fd8c8d37ce0304d4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-111.ec2.internal" Jun 11 16:08:44.794676 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:44.794551 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3023c529b9a574aede626b35cf1e00da-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal\" (UID: \"3023c529b9a574aede626b35cf1e00da\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal" Jun 11 16:08:44.814911 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.814887 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-111.ec2.internal\" not found" Jun 11 16:08:44.915573 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:44.915543 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-111.ec2.internal\" not found" Jun 11 16:08:45.002553 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.002527 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal" Jun 11 16:08:45.007305 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.007287 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-111.ec2.internal" Jun 11 16:08:45.016224 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:45.016203 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-111.ec2.internal\" not found" Jun 11 16:08:45.116799 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:45.116704 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-111.ec2.internal\" not found" Jun 11 16:08:45.217317 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:45.217281 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-111.ec2.internal\" not found" Jun 11 16:08:45.317785 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:45.317749 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-111.ec2.internal\" not found" Jun 11 16:08:45.401564 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.401541 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 16:08:45.411387 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.411367 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 16:08:45.427547 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.427527 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 11 16:08:45.427677 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.427657 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 11 16:08:45.427718 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.427665 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 11 16:08:45.464371 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.464345 2573 apiserver.go:52] "Watching apiserver" Jun 11 16:08:45.472355 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.472331 2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 11 16:08:45.472743 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.472652 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-ds2tm","openshift-image-registry/node-ca-gmrpd","openshift-multus/multus-s6txn","openshift-network-diagnostics/network-check-target-wxppc","kube-system/konnectivity-agent-s6bfw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn","openshift-dns/node-resolver-k25qj","openshift-multus/multus-additional-cni-plugins-t8tpf","openshift-multus/network-metrics-daemon-hnstd","openshift-network-operator/iptables-alerter-fwc2s","openshift-ovn-kubernetes/ovnkube-node-p7nvz","kube-system/global-pull-secret-syncer-tkm6l"] Jun 11 16:08:45.475680 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.475656 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.477760 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.477739 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gmrpd" Jun 11 16:08:45.479586 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.479561 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 11 16:08:45.479586 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.479573 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 11 16:08:45.479586 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.479575 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 11 16:08:45.479806 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.479784 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-s77mk\"" Jun 11 16:08:45.479897 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.479883 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.479986 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.479958 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w6frk\"" Jun 11 16:08:45.480036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.479993 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 11 16:08:45.480036 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.480014 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 11 16:08:45.482193 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.482174 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s6bfw" Jun 11 16:08:45.482573 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.482559 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 11 16:08:45.482728 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.482717 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 11 16:08:45.483897 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.483879 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 11 16:08:45.484198 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.484184 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5khmn\"" Jun 11 16:08:45.484350 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.484334 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:08:45.484407 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.484376 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lrcbg\"" Jun 11 16:08:45.484445 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:45.484400 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:08:45.484498 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.484475 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 11 16:08:45.484567 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.484536 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 11 16:08:45.485458 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.485445 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 11 16:08:45.486608 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.486594 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.488808 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.488793 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:08:45.488877 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:45.488855 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:08:45.489358 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.489345 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 11 16:08:45.489687 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.489633 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 11 16:08:45.489769 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.489694 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 11 16:08:45.489823 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.489786 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vdnpw\"" Jun 11 16:08:45.490154 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.490136 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 11 16:08:45.490154 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.490150 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 11 16:08:45.490282 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.490257 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 11 16:08:45.491143 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.491125 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.491366 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.491348 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 11 16:08:45.491554 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.491537 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal" Jun 11 16:08:45.492784 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.492766 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 11 16:08:45.493109 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.493094 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 11 16:08:45.493313 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.493299 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 11 16:08:45.493388 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.493339 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gztfr\"" Jun 11 16:08:45.493697 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.493682 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k25qj" Jun 11 16:08:45.495603 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.495583 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 11 16:08:45.495692 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.495635 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g2xvx\"" Jun 11 16:08:45.496366 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.496349 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.496625 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.496608 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 11 16:08:45.498148 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498130 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 11 16:08:45.498228 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498193 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-sysctl-conf\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.498228 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498212 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-host\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.498305 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498237 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e9654c3d-7ced-4f3a-8eca-9de0abf92ae3-konnectivity-ca\") pod \"konnectivity-agent-s6bfw\" (UID: \"e9654c3d-7ced-4f3a-8eca-9de0abf92ae3\") " pod="kube-system/konnectivity-agent-s6bfw" Jun 11 16:08:45.498305 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498257 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-run-netns\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.498305 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498272 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7acea643-2978-4893-a6a9-bf88a11b531b-ovnkube-config\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.498305 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498292 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-lib-modules\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.498472 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498322 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-os-release\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.498472 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498353 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-var-lib-cni-multus\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.498472 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498382 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-registration-dir\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.498472 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498403 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-etc-selinux\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.498472 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498418 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7acea643-2978-4893-a6a9-bf88a11b531b-ovn-node-metrics-cert\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.498472 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498431 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-run\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.498472 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498453 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 11 16:08:45.498472 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498456 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-var-lib-kubelet\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.498827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498492 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djcbc\" (UniqueName: \"kubernetes.io/projected/c5588305-38f6-4734-8cb8-fd14ec235f72-kube-api-access-djcbc\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.498827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498509 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w472t\" (UniqueName: \"kubernetes.io/projected/beb5dfd5-f33d-43d9-9b23-7c025879f438-kube-api-access-w472t\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.498827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498523 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sj4h\" (UniqueName: \"kubernetes.io/projected/17c6b2e4-8351-4bab-a83a-53b33503505d-kube-api-access-9sj4h\") pod \"node-ca-gmrpd\" (UID: \"17c6b2e4-8351-4bab-a83a-53b33503505d\") " pod="openshift-image-registry/node-ca-gmrpd" Jun 11 16:08:45.498827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498545 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-cnibin\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.498827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498561 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-slash\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.498827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498574 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-node-log\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.498827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498595 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-var-lib-kubelet\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.498827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498614 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-tuned\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.498827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498628 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-sys-fs\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.498827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498643 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-multus-socket-dir-parent\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.498827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498657 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qhsp7\"" Jun 11 16:08:45.498827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498658 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-log-socket\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.498827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498704 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7acea643-2978-4893-a6a9-bf88a11b531b-env-overrides\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.498827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498718 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85d4cd35-2225-4a31-b023-336e1ccbbdc7-tmp-dir\") pod \"node-resolver-k25qj\" (UID: \"85d4cd35-2225-4a31-b023-336e1ccbbdc7\") " pod="openshift-dns/node-resolver-k25qj" Jun 11 16:08:45.498827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498737 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17c6b2e4-8351-4bab-a83a-53b33503505d-host\") pod \"node-ca-gmrpd\" (UID: \"17c6b2e4-8351-4bab-a83a-53b33503505d\") " pod="openshift-image-registry/node-ca-gmrpd" Jun 11 16:08:45.498827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498753 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17c6b2e4-8351-4bab-a83a-53b33503505d-serviceca\") pod \"node-ca-gmrpd\" (UID: \"17c6b2e4-8351-4bab-a83a-53b33503505d\") " pod="openshift-image-registry/node-ca-gmrpd" Jun 11 16:08:45.498827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498774 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-run-k8s-cni-cncf-io\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.499570 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498795 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-var-lib-cni-bin\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.499570 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498819 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7acea643-2978-4893-a6a9-bf88a11b531b-ovnkube-script-lib\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.499570 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498858 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-hostroot\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.499570 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498889 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-kubelet\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.499570 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498914 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-multus-cni-dir\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.499570 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498955 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-device-dir\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.499570 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.498986 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-sysconfig\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.499570 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499008 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-sysctl-d\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.499570 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499031 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/85d4cd35-2225-4a31-b023-336e1ccbbdc7-hosts-file\") pod \"node-resolver-k25qj\" (UID: \"85d4cd35-2225-4a31-b023-336e1ccbbdc7\") " pod="openshift-dns/node-resolver-k25qj" Jun 11 16:08:45.499570 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499054 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.499570 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499074 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-modprobe-d\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.499570 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499109 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-run-netns\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.499570 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499133 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-run-multus-certs\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.499570 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499157 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-etc-kubernetes\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.499570 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499181 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs\") pod \"network-metrics-daemon-hnstd\" (UID: \"7cc2d1d8-23c4-462c-854b-c005f8e29363\") " pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:08:45.499570 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499210 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-run-ovn\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.499570 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499242 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-var-lib-openvswitch\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.500230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499272 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fwc2s" Jun 11 16:08:45.500230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499271 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-run-ovn-kubernetes\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.500230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499571 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/764eca77-76b0-4ef2-a59f-ad632d07aeac-cni-binary-copy\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.500230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499604 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-multus-conf-dir\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.500230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499671 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/764eca77-76b0-4ef2-a59f-ad632d07aeac-multus-daemon-config\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.500230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499702 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jphx9\" (UniqueName: \"kubernetes.io/projected/764eca77-76b0-4ef2-a59f-ad632d07aeac-kube-api-access-jphx9\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.500230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499726 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e9654c3d-7ced-4f3a-8eca-9de0abf92ae3-agent-certs\") pod \"konnectivity-agent-s6bfw\" (UID: \"e9654c3d-7ced-4f3a-8eca-9de0abf92ae3\") " pod="kube-system/konnectivity-agent-s6bfw" Jun 11 16:08:45.500230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499748 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-run-systemd\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.500230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499780 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-etc-openvswitch\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.500230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499798 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.500230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499823 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv77p\" (UniqueName: \"kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p\") pod \"network-check-target-wxppc\" (UID: \"d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76\") " pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:08:45.500230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499850 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-systemd\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.500230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499877 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-sys\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.500230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499902 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndc4f\" (UniqueName: \"kubernetes.io/projected/7cc2d1d8-23c4-462c-854b-c005f8e29363-kube-api-access-ndc4f\") pod \"network-metrics-daemon-hnstd\" (UID: \"7cc2d1d8-23c4-462c-854b-c005f8e29363\") " pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:08:45.500230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499944 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-system-cni-dir\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.500230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499967 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-systemd-units\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.500230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499984 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-cni-bin\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.500790 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.499998 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-socket-dir\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.500790 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.500014 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/beb5dfd5-f33d-43d9-9b23-7c025879f438-tmp\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.500790 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.500074 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krv7l\" (UniqueName: \"kubernetes.io/projected/85d4cd35-2225-4a31-b023-336e1ccbbdc7-kube-api-access-krv7l\") pod \"node-resolver-k25qj\" (UID: \"85d4cd35-2225-4a31-b023-336e1ccbbdc7\") " pod="openshift-dns/node-resolver-k25qj" Jun 11 16:08:45.500790 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.500105 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-run-openvswitch\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.500790 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.500156 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-cni-netd\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.500790 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.500173 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gqqh\" (UniqueName: \"kubernetes.io/projected/7acea643-2978-4893-a6a9-bf88a11b531b-kube-api-access-5gqqh\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.500790 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.500191 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-kubernetes\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.501183 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.501167 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 11 16:08:45.501364 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.501351 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jgkxs\"" Jun 11 16:08:45.501364 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.501359 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 11 16:08:45.501695 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.501682 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 11 16:08:45.501969 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.501953 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:45.502068 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:45.502050 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:08:45.507434 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.507417 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal"] Jun 11 16:08:45.507537 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.507525 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 16:08:45.508322 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.508140 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-111.ec2.internal" Jun 11 16:08:45.509079 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.508846 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 11 16:08:45.511783 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:45.511749 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3154ce17f76996f6fd8c8d37ce0304d4.slice/crio-16be86f886934fd41939427758287e8042d499cc88936606e2e96b592de91caa WatchSource:0}: Error finding container 16be86f886934fd41939427758287e8042d499cc88936606e2e96b592de91caa: Status 404 returned error can't find the container with id 16be86f886934fd41939427758287e8042d499cc88936606e2e96b592de91caa Jun 11 16:08:45.512572 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:45.512552 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3023c529b9a574aede626b35cf1e00da.slice/crio-a07f7ef62f36173d63500081b079dc3e78b3822c07196e86e46d99635754f6d1 WatchSource:0}: Error finding container a07f7ef62f36173d63500081b079dc3e78b3822c07196e86e46d99635754f6d1: Status 404 returned error can't find the container with id a07f7ef62f36173d63500081b079dc3e78b3822c07196e86e46d99635754f6d1 Jun 11 16:08:45.517047 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.517033 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 16:08:45.517115 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.517085 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-111.ec2.internal"] Jun 11 16:08:45.517262 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.517241 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 11 16:08:45.549721 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.549702 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-p2vrc" Jun 11 16:08:45.565860 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.565821 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-111.ec2.internal" event={"ID":"3154ce17f76996f6fd8c8d37ce0304d4","Type":"ContainerStarted","Data":"16be86f886934fd41939427758287e8042d499cc88936606e2e96b592de91caa"} Jun 11 16:08:45.566776 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.566757 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal" event={"ID":"3023c529b9a574aede626b35cf1e00da","Type":"ContainerStarted","Data":"a07f7ef62f36173d63500081b079dc3e78b3822c07196e86e46d99635754f6d1"} Jun 11 16:08:45.568035 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.568015 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-p2vrc" Jun 11 16:08:45.571235 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.571193 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-10 16:03:44 +0000 UTC" deadline="2028-01-29 09:58:04.398046053 +0000 UTC" Jun 11 16:08:45.571300 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.571234 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14321h49m18.826813955s" Jun 11 16:08:45.592904 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.592884 2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 11 16:08:45.600636 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600614 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7acea643-2978-4893-a6a9-bf88a11b531b-ovnkube-script-lib\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.600703 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600642 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9669133-a0d2-475c-85c1-7ad84b9a59ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.600703 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600660 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-hostroot\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.600703 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600675 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-kubelet\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.600703 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600700 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a412864-1268-4787-be57-d25fd2e7340d-dbus\") pod \"global-pull-secret-syncer-tkm6l\" (UID: \"5a412864-1268-4787-be57-d25fd2e7340d\") " pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:45.600892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600718 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-multus-cni-dir\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.600892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600741 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-device-dir\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.600892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600739 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-hostroot\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.600892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600752 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-kubelet\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.600892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600763 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-sysconfig\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.600892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600799 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-sysctl-d\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.600892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600803 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-multus-cni-dir\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.600892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600799 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-device-dir\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.600892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600823 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/85d4cd35-2225-4a31-b023-336e1ccbbdc7-hosts-file\") pod \"node-resolver-k25qj\" (UID: \"85d4cd35-2225-4a31-b023-336e1ccbbdc7\") " pod="openshift-dns/node-resolver-k25qj" Jun 11 16:08:45.600892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600845 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-sysconfig\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.600892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600848 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9669133-a0d2-475c-85c1-7ad84b9a59ba-system-cni-dir\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.600892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600886 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.600892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600888 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/85d4cd35-2225-4a31-b023-336e1ccbbdc7-hosts-file\") pod \"node-resolver-k25qj\" (UID: \"85d4cd35-2225-4a31-b023-336e1ccbbdc7\") " pod="openshift-dns/node-resolver-k25qj" Jun 11 16:08:45.600892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600901 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-modprobe-d\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600942 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-sysctl-d\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600951 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600945 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-run-netns\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600973 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-run-netns\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.600986 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-run-multus-certs\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601011 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-etc-kubernetes\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601014 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-modprobe-d\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601027 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs\") pod \"network-metrics-daemon-hnstd\" (UID: \"7cc2d1d8-23c4-462c-854b-c005f8e29363\") " pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601043 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-run-ovn\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601046 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-run-multus-certs\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601063 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-var-lib-openvswitch\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601079 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-etc-kubernetes\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601088 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-run-ovn-kubernetes\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601099 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-run-ovn\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601114 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/764eca77-76b0-4ef2-a59f-ad632d07aeac-cni-binary-copy\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601122 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-var-lib-openvswitch\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601142 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-multus-conf-dir\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.601900 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601148 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-run-ovn-kubernetes\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.602640 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601167 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/764eca77-76b0-4ef2-a59f-ad632d07aeac-multus-daemon-config\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.602640 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601193 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jphx9\" (UniqueName: \"kubernetes.io/projected/764eca77-76b0-4ef2-a59f-ad632d07aeac-kube-api-access-jphx9\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.602640 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:45.601199 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:08:45.602640 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601219 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e9654c3d-7ced-4f3a-8eca-9de0abf92ae3-agent-certs\") pod \"konnectivity-agent-s6bfw\" (UID: \"e9654c3d-7ced-4f3a-8eca-9de0abf92ae3\") " pod="kube-system/konnectivity-agent-s6bfw" Jun 11 16:08:45.602640 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601238 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-run-systemd\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.602640 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601258 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-etc-openvswitch\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.602640 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601186 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-multus-conf-dir\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.602640 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601289 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.602640 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601322 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-etc-openvswitch\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.602640 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601322 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv77p\" (UniqueName: \"kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p\") pod \"network-check-target-wxppc\" (UID: \"d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76\") " pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:08:45.602640 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601366 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-systemd\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.602640 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601376 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7acea643-2978-4893-a6a9-bf88a11b531b-ovnkube-script-lib\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.602640 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601390 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-sys\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.602640 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601401 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.602640 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601433 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-sys\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.602640 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601464 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-run-systemd\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.602640 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:45.601485 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs podName:7cc2d1d8-23c4-462c-854b-c005f8e29363 nodeName:}" failed. No retries permitted until 2026-06-11 16:08:46.101386287 +0000 UTC m=+1.935498804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs") pod "network-metrics-daemon-hnstd" (UID: "7cc2d1d8-23c4-462c-854b-c005f8e29363") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:08:45.603647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601487 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-systemd\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.603647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601616 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9669133-a0d2-475c-85c1-7ad84b9a59ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.603647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601643 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whjz8\" (UniqueName: \"kubernetes.io/projected/5010e752-7999-4b90-bba6-6423eab6f4f2-kube-api-access-whjz8\") pod \"iptables-alerter-fwc2s\" (UID: \"5010e752-7999-4b90-bba6-6423eab6f4f2\") " pod="openshift-network-operator/iptables-alerter-fwc2s" Jun 11 16:08:45.603647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601665 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a412864-1268-4787-be57-d25fd2e7340d-kubelet-config\") pod \"global-pull-secret-syncer-tkm6l\" (UID: \"5a412864-1268-4787-be57-d25fd2e7340d\") " pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:45.603647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601719 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndc4f\" (UniqueName: \"kubernetes.io/projected/7cc2d1d8-23c4-462c-854b-c005f8e29363-kube-api-access-ndc4f\") pod \"network-metrics-daemon-hnstd\" (UID: \"7cc2d1d8-23c4-462c-854b-c005f8e29363\") " pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:08:45.603647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601746 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/764eca77-76b0-4ef2-a59f-ad632d07aeac-cni-binary-copy\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.603647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601769 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-system-cni-dir\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.603647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601798 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-systemd-units\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.603647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601815 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-cni-bin\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.603647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601817 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 11 16:08:45.603647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601830 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-socket-dir\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.603647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601873 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/beb5dfd5-f33d-43d9-9b23-7c025879f438-tmp\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.603647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601897 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krv7l\" (UniqueName: \"kubernetes.io/projected/85d4cd35-2225-4a31-b023-336e1ccbbdc7-kube-api-access-krv7l\") pod \"node-resolver-k25qj\" (UID: \"85d4cd35-2225-4a31-b023-336e1ccbbdc7\") " pod="openshift-dns/node-resolver-k25qj" Jun 11 16:08:45.603647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601891 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-systemd-units\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.603647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601946 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-system-cni-dir\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.603647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601941 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-run-openvswitch\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.603647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601949 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/764eca77-76b0-4ef2-a59f-ad632d07aeac-multus-daemon-config\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.604438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.601993 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-socket-dir\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.604438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602004 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-run-openvswitch\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.604438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602032 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-cni-bin\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.604438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602047 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-cni-netd\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.604438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602074 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gqqh\" (UniqueName: \"kubernetes.io/projected/7acea643-2978-4893-a6a9-bf88a11b531b-kube-api-access-5gqqh\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.604438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602089 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-cni-netd\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.604438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602100 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-kubernetes\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.604438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602122 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-sysctl-conf\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.604438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602149 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-host\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.604438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602175 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5010e752-7999-4b90-bba6-6423eab6f4f2-host-slash\") pod \"iptables-alerter-fwc2s\" (UID: \"5010e752-7999-4b90-bba6-6423eab6f4f2\") " pod="openshift-network-operator/iptables-alerter-fwc2s" Jun 11 16:08:45.604438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602177 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-kubernetes\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.604438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602200 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e9654c3d-7ced-4f3a-8eca-9de0abf92ae3-konnectivity-ca\") pod \"konnectivity-agent-s6bfw\" (UID: \"e9654c3d-7ced-4f3a-8eca-9de0abf92ae3\") " pod="kube-system/konnectivity-agent-s6bfw" Jun 11 16:08:45.604438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602231 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-run-netns\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.604438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602250 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-host\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.604438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602255 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7acea643-2978-4893-a6a9-bf88a11b531b-ovnkube-config\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.604438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602292 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-lib-modules\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.604438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602295 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-sysctl-conf\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.605203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602303 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-run-netns\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.605203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602353 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-os-release\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.605203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602392 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-var-lib-cni-multus\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.605203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602412 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-os-release\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.605203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602421 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-registration-dir\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.605203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602447 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9669133-a0d2-475c-85c1-7ad84b9a59ba-cnibin\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.605203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602452 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-var-lib-cni-multus\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.605203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602478 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-etc-selinux\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.605203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602496 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-lib-modules\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.605203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602501 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-registration-dir\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.605203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602503 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7acea643-2978-4893-a6a9-bf88a11b531b-ovn-node-metrics-cert\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.605203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602615 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-run\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.605203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602662 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-run\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.605203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602698 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e9654c3d-7ced-4f3a-8eca-9de0abf92ae3-konnectivity-ca\") pod \"konnectivity-agent-s6bfw\" (UID: \"e9654c3d-7ced-4f3a-8eca-9de0abf92ae3\") " pod="kube-system/konnectivity-agent-s6bfw" Jun 11 16:08:45.605203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602693 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9669133-a0d2-475c-85c1-7ad84b9a59ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.605203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602686 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-etc-selinux\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.605203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602722 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7acea643-2978-4893-a6a9-bf88a11b531b-ovnkube-config\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.605755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602747 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9669133-a0d2-475c-85c1-7ad84b9a59ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.605755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602763 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sj28\" (UniqueName: \"kubernetes.io/projected/b9669133-a0d2-475c-85c1-7ad84b9a59ba-kube-api-access-5sj28\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.605755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602786 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-var-lib-kubelet\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.605755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602880 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-var-lib-kubelet\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.605755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602833 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djcbc\" (UniqueName: \"kubernetes.io/projected/c5588305-38f6-4734-8cb8-fd14ec235f72-kube-api-access-djcbc\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.605755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602950 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w472t\" (UniqueName: \"kubernetes.io/projected/beb5dfd5-f33d-43d9-9b23-7c025879f438-kube-api-access-w472t\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.605755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.602975 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b9669133-a0d2-475c-85c1-7ad84b9a59ba-os-release\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.605755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603006 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret\") pod \"global-pull-secret-syncer-tkm6l\" (UID: \"5a412864-1268-4787-be57-d25fd2e7340d\") " pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:45.605755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603030 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sj4h\" (UniqueName: \"kubernetes.io/projected/17c6b2e4-8351-4bab-a83a-53b33503505d-kube-api-access-9sj4h\") pod \"node-ca-gmrpd\" (UID: \"17c6b2e4-8351-4bab-a83a-53b33503505d\") " pod="openshift-image-registry/node-ca-gmrpd" Jun 11 16:08:45.605755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603052 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-cnibin\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.605755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603076 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-slash\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.605755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603098 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-node-log\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.605755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603125 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-cnibin\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.605755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603144 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-var-lib-kubelet\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.605755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603168 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-tuned\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.605755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603194 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-sys-fs\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.605755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603199 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-node-log\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603216 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-multus-socket-dir-parent\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603239 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-log-socket\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603261 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7acea643-2978-4893-a6a9-bf88a11b531b-env-overrides\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603283 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85d4cd35-2225-4a31-b023-336e1ccbbdc7-tmp-dir\") pod \"node-resolver-k25qj\" (UID: \"85d4cd35-2225-4a31-b023-336e1ccbbdc7\") " pod="openshift-dns/node-resolver-k25qj" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603302 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c5588305-38f6-4734-8cb8-fd14ec235f72-sys-fs\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603170 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-host-slash\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603333 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5010e752-7999-4b90-bba6-6423eab6f4f2-iptables-alerter-script\") pod \"iptables-alerter-fwc2s\" (UID: \"5010e752-7999-4b90-bba6-6423eab6f4f2\") " pod="openshift-network-operator/iptables-alerter-fwc2s" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603369 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/beb5dfd5-f33d-43d9-9b23-7c025879f438-var-lib-kubelet\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603390 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17c6b2e4-8351-4bab-a83a-53b33503505d-host\") pod \"node-ca-gmrpd\" (UID: \"17c6b2e4-8351-4bab-a83a-53b33503505d\") " pod="openshift-image-registry/node-ca-gmrpd" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603425 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17c6b2e4-8351-4bab-a83a-53b33503505d-serviceca\") pod \"node-ca-gmrpd\" (UID: \"17c6b2e4-8351-4bab-a83a-53b33503505d\") " pod="openshift-image-registry/node-ca-gmrpd" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603424 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7acea643-2978-4893-a6a9-bf88a11b531b-log-socket\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603483 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-multus-socket-dir-parent\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603489 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17c6b2e4-8351-4bab-a83a-53b33503505d-host\") pod \"node-ca-gmrpd\" (UID: \"17c6b2e4-8351-4bab-a83a-53b33503505d\") " pod="openshift-image-registry/node-ca-gmrpd" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603530 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-run-k8s-cni-cncf-io\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603560 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-var-lib-cni-bin\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603650 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-var-lib-cni-bin\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603684 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85d4cd35-2225-4a31-b023-336e1ccbbdc7-tmp-dir\") pod \"node-resolver-k25qj\" (UID: \"85d4cd35-2225-4a31-b023-336e1ccbbdc7\") " pod="openshift-dns/node-resolver-k25qj" Jun 11 16:08:45.606247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603691 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/764eca77-76b0-4ef2-a59f-ad632d07aeac-host-run-k8s-cni-cncf-io\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.606728 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.603781 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7acea643-2978-4893-a6a9-bf88a11b531b-env-overrides\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.606728 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.604127 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17c6b2e4-8351-4bab-a83a-53b33503505d-serviceca\") pod \"node-ca-gmrpd\" (UID: \"17c6b2e4-8351-4bab-a83a-53b33503505d\") " pod="openshift-image-registry/node-ca-gmrpd" Jun 11 16:08:45.606728 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.605295 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/beb5dfd5-f33d-43d9-9b23-7c025879f438-etc-tuned\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.606728 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.605415 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/beb5dfd5-f33d-43d9-9b23-7c025879f438-tmp\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.606728 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.605511 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7acea643-2978-4893-a6a9-bf88a11b531b-ovn-node-metrics-cert\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.606728 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.605642 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e9654c3d-7ced-4f3a-8eca-9de0abf92ae3-agent-certs\") pod \"konnectivity-agent-s6bfw\" (UID: \"e9654c3d-7ced-4f3a-8eca-9de0abf92ae3\") " pod="kube-system/konnectivity-agent-s6bfw" Jun 11 16:08:45.609367 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:45.609348 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:08:45.609410 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:45.609369 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:08:45.609410 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:45.609394 2573 projected.go:196] Error preparing data for projected volume kube-api-access-sv77p for pod openshift-network-diagnostics/network-check-target-wxppc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:08:45.609479 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:45.609465 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p podName:d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76 nodeName:}" failed. No retries permitted until 2026-06-11 16:08:46.109449091 +0000 UTC m=+1.943561620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sv77p" (UniqueName: "kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p") pod "network-check-target-wxppc" (UID: "d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:08:45.611028 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.611013 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gqqh\" (UniqueName: \"kubernetes.io/projected/7acea643-2978-4893-a6a9-bf88a11b531b-kube-api-access-5gqqh\") pod \"ovnkube-node-p7nvz\" (UID: \"7acea643-2978-4893-a6a9-bf88a11b531b\") " pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.613842 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.613335 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndc4f\" (UniqueName: \"kubernetes.io/projected/7cc2d1d8-23c4-462c-854b-c005f8e29363-kube-api-access-ndc4f\") pod \"network-metrics-daemon-hnstd\" (UID: \"7cc2d1d8-23c4-462c-854b-c005f8e29363\") " pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:08:45.614965 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.614916 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krv7l\" (UniqueName: \"kubernetes.io/projected/85d4cd35-2225-4a31-b023-336e1ccbbdc7-kube-api-access-krv7l\") pod \"node-resolver-k25qj\" (UID: \"85d4cd35-2225-4a31-b023-336e1ccbbdc7\") " pod="openshift-dns/node-resolver-k25qj" Jun 11 16:08:45.615875 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.615852 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w472t\" (UniqueName: \"kubernetes.io/projected/beb5dfd5-f33d-43d9-9b23-7c025879f438-kube-api-access-w472t\") pod \"tuned-ds2tm\" (UID: \"beb5dfd5-f33d-43d9-9b23-7c025879f438\") " pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.616131 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.616113 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sj4h\" (UniqueName: \"kubernetes.io/projected/17c6b2e4-8351-4bab-a83a-53b33503505d-kube-api-access-9sj4h\") pod \"node-ca-gmrpd\" (UID: \"17c6b2e4-8351-4bab-a83a-53b33503505d\") " pod="openshift-image-registry/node-ca-gmrpd" Jun 11 16:08:45.616131 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.616126 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djcbc\" (UniqueName: \"kubernetes.io/projected/c5588305-38f6-4734-8cb8-fd14ec235f72-kube-api-access-djcbc\") pod \"aws-ebs-csi-driver-node-8clfn\" (UID: \"c5588305-38f6-4734-8cb8-fd14ec235f72\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.616268 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.616252 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jphx9\" (UniqueName: \"kubernetes.io/projected/764eca77-76b0-4ef2-a59f-ad632d07aeac-kube-api-access-jphx9\") pod \"multus-s6txn\" (UID: \"764eca77-76b0-4ef2-a59f-ad632d07aeac\") " pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.704600 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704517 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9669133-a0d2-475c-85c1-7ad84b9a59ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.704600 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704555 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whjz8\" (UniqueName: \"kubernetes.io/projected/5010e752-7999-4b90-bba6-6423eab6f4f2-kube-api-access-whjz8\") pod \"iptables-alerter-fwc2s\" (UID: \"5010e752-7999-4b90-bba6-6423eab6f4f2\") " pod="openshift-network-operator/iptables-alerter-fwc2s" Jun 11 16:08:45.704600 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704578 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a412864-1268-4787-be57-d25fd2e7340d-kubelet-config\") pod \"global-pull-secret-syncer-tkm6l\" (UID: \"5a412864-1268-4787-be57-d25fd2e7340d\") " pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:45.704845 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704610 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5010e752-7999-4b90-bba6-6423eab6f4f2-host-slash\") pod \"iptables-alerter-fwc2s\" (UID: \"5010e752-7999-4b90-bba6-6423eab6f4f2\") " pod="openshift-network-operator/iptables-alerter-fwc2s" Jun 11 16:08:45.704845 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704636 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9669133-a0d2-475c-85c1-7ad84b9a59ba-cnibin\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.704845 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704659 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9669133-a0d2-475c-85c1-7ad84b9a59ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.704845 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704666 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5a412864-1268-4787-be57-d25fd2e7340d-kubelet-config\") pod \"global-pull-secret-syncer-tkm6l\" (UID: \"5a412864-1268-4787-be57-d25fd2e7340d\") " pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:45.704845 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704681 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9669133-a0d2-475c-85c1-7ad84b9a59ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.704845 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704725 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sj28\" (UniqueName: \"kubernetes.io/projected/b9669133-a0d2-475c-85c1-7ad84b9a59ba-kube-api-access-5sj28\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.704845 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704743 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b9669133-a0d2-475c-85c1-7ad84b9a59ba-cnibin\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.704845 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704757 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b9669133-a0d2-475c-85c1-7ad84b9a59ba-os-release\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.704845 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704774 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5010e752-7999-4b90-bba6-6423eab6f4f2-host-slash\") pod \"iptables-alerter-fwc2s\" (UID: \"5010e752-7999-4b90-bba6-6423eab6f4f2\") " pod="openshift-network-operator/iptables-alerter-fwc2s" Jun 11 16:08:45.704845 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704797 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret\") pod \"global-pull-secret-syncer-tkm6l\" (UID: \"5a412864-1268-4787-be57-d25fd2e7340d\") " pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:45.704845 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704818 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b9669133-a0d2-475c-85c1-7ad84b9a59ba-os-release\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.704845 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704834 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5010e752-7999-4b90-bba6-6423eab6f4f2-iptables-alerter-script\") pod \"iptables-alerter-fwc2s\" (UID: \"5010e752-7999-4b90-bba6-6423eab6f4f2\") " pod="openshift-network-operator/iptables-alerter-fwc2s" Jun 11 16:08:45.705449 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704861 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9669133-a0d2-475c-85c1-7ad84b9a59ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.705449 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704883 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a412864-1268-4787-be57-d25fd2e7340d-dbus\") pod \"global-pull-secret-syncer-tkm6l\" (UID: \"5a412864-1268-4787-be57-d25fd2e7340d\") " pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:45.705449 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:45.704894 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:08:45.705449 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704908 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9669133-a0d2-475c-85c1-7ad84b9a59ba-system-cni-dir\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.705449 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:45.704979 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret podName:5a412864-1268-4787-be57-d25fd2e7340d nodeName:}" failed. No retries permitted until 2026-06-11 16:08:46.204960846 +0000 UTC m=+2.039073363 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret") pod "global-pull-secret-syncer-tkm6l" (UID: "5a412864-1268-4787-be57-d25fd2e7340d") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:08:45.705449 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.704973 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b9669133-a0d2-475c-85c1-7ad84b9a59ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.705449 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.705033 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b9669133-a0d2-475c-85c1-7ad84b9a59ba-system-cni-dir\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.705449 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.705095 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5a412864-1268-4787-be57-d25fd2e7340d-dbus\") pod \"global-pull-secret-syncer-tkm6l\" (UID: \"5a412864-1268-4787-be57-d25fd2e7340d\") " pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:45.705449 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.705206 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b9669133-a0d2-475c-85c1-7ad84b9a59ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.705449 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.705410 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5010e752-7999-4b90-bba6-6423eab6f4f2-iptables-alerter-script\") pod \"iptables-alerter-fwc2s\" (UID: \"5010e752-7999-4b90-bba6-6423eab6f4f2\") " pod="openshift-network-operator/iptables-alerter-fwc2s" Jun 11 16:08:45.705804 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.705496 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b9669133-a0d2-475c-85c1-7ad84b9a59ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.705995 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.705967 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b9669133-a0d2-475c-85c1-7ad84b9a59ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.717375 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.717352 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whjz8\" (UniqueName: \"kubernetes.io/projected/5010e752-7999-4b90-bba6-6423eab6f4f2-kube-api-access-whjz8\") pod \"iptables-alerter-fwc2s\" (UID: \"5010e752-7999-4b90-bba6-6423eab6f4f2\") " pod="openshift-network-operator/iptables-alerter-fwc2s" Jun 11 16:08:45.717982 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.717966 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sj28\" (UniqueName: \"kubernetes.io/projected/b9669133-a0d2-475c-85c1-7ad84b9a59ba-kube-api-access-5sj28\") pod \"multus-additional-cni-plugins-t8tpf\" (UID: \"b9669133-a0d2-475c-85c1-7ad84b9a59ba\") " pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.811256 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.811230 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" Jun 11 16:08:45.817772 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.817751 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gmrpd" Jun 11 16:08:45.822068 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:45.822041 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbeb5dfd5_f33d_43d9_9b23_7c025879f438.slice/crio-f0ddef8180d2e6ea442b4a8d886c35cc7296c22b58118ff4284fa907c43a4c14 WatchSource:0}: Error finding container f0ddef8180d2e6ea442b4a8d886c35cc7296c22b58118ff4284fa907c43a4c14: Status 404 returned error can't find the container with id f0ddef8180d2e6ea442b4a8d886c35cc7296c22b58118ff4284fa907c43a4c14 Jun 11 16:08:45.825696 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.825676 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s6txn" Jun 11 16:08:45.827433 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:45.827408 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17c6b2e4_8351_4bab_a83a_53b33503505d.slice/crio-42af50edbfe2e4cab8cc6aafeeb3c5a95c9e7b4216157bd70c0a233afe8a781f WatchSource:0}: Error finding container 42af50edbfe2e4cab8cc6aafeeb3c5a95c9e7b4216157bd70c0a233afe8a781f: Status 404 returned error can't find the container with id 42af50edbfe2e4cab8cc6aafeeb3c5a95c9e7b4216157bd70c0a233afe8a781f Jun 11 16:08:45.830286 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.830268 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s6bfw" Jun 11 16:08:45.834632 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:45.834614 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod764eca77_76b0_4ef2_a59f_ad632d07aeac.slice/crio-51f65b68f1d1066a4d1831f8e68571c4421850c5df16c3ff5b1d28c7cfeb4355 WatchSource:0}: Error finding container 51f65b68f1d1066a4d1831f8e68571c4421850c5df16c3ff5b1d28c7cfeb4355: Status 404 returned error can't find the container with id 51f65b68f1d1066a4d1831f8e68571c4421850c5df16c3ff5b1d28c7cfeb4355 Jun 11 16:08:45.836811 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.836796 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:08:45.840222 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:45.840195 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9654c3d_7ced_4f3a_8eca_9de0abf92ae3.slice/crio-0ab7b03a0c14993e92200e97924ba40a79d16df5e6b5046665ca557c246b41d0 WatchSource:0}: Error finding container 0ab7b03a0c14993e92200e97924ba40a79d16df5e6b5046665ca557c246b41d0: Status 404 returned error can't find the container with id 0ab7b03a0c14993e92200e97924ba40a79d16df5e6b5046665ca557c246b41d0 Jun 11 16:08:45.841336 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.841304 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" Jun 11 16:08:45.845393 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:45.845377 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7acea643_2978_4893_a6a9_bf88a11b531b.slice/crio-1f59a71e8a44c09b868243a15f41e8785e89c9eab31236a5785b4cac5827c29c WatchSource:0}: Error finding container 1f59a71e8a44c09b868243a15f41e8785e89c9eab31236a5785b4cac5827c29c: Status 404 returned error can't find the container with id 1f59a71e8a44c09b868243a15f41e8785e89c9eab31236a5785b4cac5827c29c Jun 11 16:08:45.846448 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.846428 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k25qj" Jun 11 16:08:45.850614 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:45.850594 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5588305_38f6_4734_8cb8_fd14ec235f72.slice/crio-c151a8823a0503aac6284b72a7bcc27d7146824ca7cf281a349646a326fed6b5 WatchSource:0}: Error finding container c151a8823a0503aac6284b72a7bcc27d7146824ca7cf281a349646a326fed6b5: Status 404 returned error can't find the container with id c151a8823a0503aac6284b72a7bcc27d7146824ca7cf281a349646a326fed6b5 Jun 11 16:08:45.851697 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.851678 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-t8tpf" Jun 11 16:08:45.856198 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:45.856179 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85d4cd35_2225_4a31_b023_336e1ccbbdc7.slice/crio-bdfd9a660eae0754a3ee297117f582d28523fada4c7215d127a04cdc961fa140 WatchSource:0}: Error finding container bdfd9a660eae0754a3ee297117f582d28523fada4c7215d127a04cdc961fa140: Status 404 returned error can't find the container with id bdfd9a660eae0754a3ee297117f582d28523fada4c7215d127a04cdc961fa140 Jun 11 16:08:45.856944 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:45.856876 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fwc2s" Jun 11 16:08:45.862101 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:45.862040 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9669133_a0d2_475c_85c1_7ad84b9a59ba.slice/crio-5f5ab09a1114662f6f9807fe516f5b07f3a35f63581e7746bf8dcdad4bf0315c WatchSource:0}: Error finding container 5f5ab09a1114662f6f9807fe516f5b07f3a35f63581e7746bf8dcdad4bf0315c: Status 404 returned error can't find the container with id 5f5ab09a1114662f6f9807fe516f5b07f3a35f63581e7746bf8dcdad4bf0315c Jun 11 16:08:45.866866 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:08:45.866847 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5010e752_7999_4b90_bba6_6423eab6f4f2.slice/crio-16b77bf49cd1c562e40dd7160f6d74fa55b25a59cdf4af22d827d1c02c9a1625 WatchSource:0}: Error finding container 16b77bf49cd1c562e40dd7160f6d74fa55b25a59cdf4af22d827d1c02c9a1625: Status 404 returned error can't find the container with id 16b77bf49cd1c562e40dd7160f6d74fa55b25a59cdf4af22d827d1c02c9a1625 Jun 11 16:08:46.015618 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:46.015536 2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 16:08:46.109341 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:46.109302 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs\") pod \"network-metrics-daemon-hnstd\" (UID: \"7cc2d1d8-23c4-462c-854b-c005f8e29363\") " pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:08:46.109524 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:46.109494 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:08:46.109582 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:46.109562 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs podName:7cc2d1d8-23c4-462c-854b-c005f8e29363 nodeName:}" failed. No retries permitted until 2026-06-11 16:08:47.109546758 +0000 UTC m=+2.943659278 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs") pod "network-metrics-daemon-hnstd" (UID: "7cc2d1d8-23c4-462c-854b-c005f8e29363") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:08:46.210158 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:46.210127 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret\") pod \"global-pull-secret-syncer-tkm6l\" (UID: \"5a412864-1268-4787-be57-d25fd2e7340d\") " pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:46.210333 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:46.210211 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv77p\" (UniqueName: \"kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p\") pod \"network-check-target-wxppc\" (UID: \"d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76\") " pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:08:46.210390 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:46.210370 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:08:46.210390 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:46.210386 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:08:46.210492 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:46.210399 2573 projected.go:196] Error preparing data for projected volume kube-api-access-sv77p for pod openshift-network-diagnostics/network-check-target-wxppc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:08:46.210492 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:46.210452 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p podName:d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76 nodeName:}" failed. No retries permitted until 2026-06-11 16:08:47.210436108 +0000 UTC m=+3.044548643 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sv77p" (UniqueName: "kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p") pod "network-check-target-wxppc" (UID: "d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:08:46.210781 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:46.210764 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:08:46.210876 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:46.210812 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret podName:5a412864-1268-4787-be57-d25fd2e7340d nodeName:}" failed. No retries permitted until 2026-06-11 16:08:47.210798241 +0000 UTC m=+3.044910772 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret") pod "global-pull-secret-syncer-tkm6l" (UID: "5a412864-1268-4787-be57-d25fd2e7340d") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:08:46.573735 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:46.572275 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 16:03:45 +0000 UTC" deadline="2028-01-16 15:34:40.186131925 +0000 UTC" Jun 11 16:08:46.573735 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:46.572334 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14015h25m53.613801732s" Jun 11 16:08:46.579864 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:46.579823 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8tpf" event={"ID":"b9669133-a0d2-475c-85c1-7ad84b9a59ba","Type":"ContainerStarted","Data":"5f5ab09a1114662f6f9807fe516f5b07f3a35f63581e7746bf8dcdad4bf0315c"} Jun 11 16:08:46.587731 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:46.587699 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k25qj" event={"ID":"85d4cd35-2225-4a31-b023-336e1ccbbdc7","Type":"ContainerStarted","Data":"bdfd9a660eae0754a3ee297117f582d28523fada4c7215d127a04cdc961fa140"} Jun 11 16:08:46.591651 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:46.591625 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" event={"ID":"c5588305-38f6-4734-8cb8-fd14ec235f72","Type":"ContainerStarted","Data":"c151a8823a0503aac6284b72a7bcc27d7146824ca7cf281a349646a326fed6b5"} Jun 11 16:08:46.600781 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:46.600751 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s6bfw" event={"ID":"e9654c3d-7ced-4f3a-8eca-9de0abf92ae3","Type":"ContainerStarted","Data":"0ab7b03a0c14993e92200e97924ba40a79d16df5e6b5046665ca557c246b41d0"} Jun 11 16:08:46.617772 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:46.617745 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gmrpd" event={"ID":"17c6b2e4-8351-4bab-a83a-53b33503505d","Type":"ContainerStarted","Data":"42af50edbfe2e4cab8cc6aafeeb3c5a95c9e7b4216157bd70c0a233afe8a781f"} Jun 11 16:08:46.625237 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:46.625200 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" event={"ID":"beb5dfd5-f33d-43d9-9b23-7c025879f438","Type":"ContainerStarted","Data":"f0ddef8180d2e6ea442b4a8d886c35cc7296c22b58118ff4284fa907c43a4c14"} Jun 11 16:08:46.628817 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:46.628795 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fwc2s" event={"ID":"5010e752-7999-4b90-bba6-6423eab6f4f2","Type":"ContainerStarted","Data":"16b77bf49cd1c562e40dd7160f6d74fa55b25a59cdf4af22d827d1c02c9a1625"} Jun 11 16:08:46.635968 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:46.635898 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" event={"ID":"7acea643-2978-4893-a6a9-bf88a11b531b","Type":"ContainerStarted","Data":"1f59a71e8a44c09b868243a15f41e8785e89c9eab31236a5785b4cac5827c29c"} Jun 11 16:08:46.648682 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:46.646100 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s6txn" event={"ID":"764eca77-76b0-4ef2-a59f-ad632d07aeac","Type":"ContainerStarted","Data":"51f65b68f1d1066a4d1831f8e68571c4421850c5df16c3ff5b1d28c7cfeb4355"} Jun 11 16:08:46.956606 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:46.956525 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 16:08:47.119640 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:47.119108 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs\") pod \"network-metrics-daemon-hnstd\" (UID: \"7cc2d1d8-23c4-462c-854b-c005f8e29363\") " pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:08:47.119640 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:47.119273 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:08:47.119640 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:47.119336 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs podName:7cc2d1d8-23c4-462c-854b-c005f8e29363 nodeName:}" failed. No retries permitted until 2026-06-11 16:08:49.119316002 +0000 UTC m=+4.953428542 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs") pod "network-metrics-daemon-hnstd" (UID: "7cc2d1d8-23c4-462c-854b-c005f8e29363") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:08:47.219779 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:47.219702 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret\") pod \"global-pull-secret-syncer-tkm6l\" (UID: \"5a412864-1268-4787-be57-d25fd2e7340d\") " pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:47.220032 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:47.219787 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv77p\" (UniqueName: \"kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p\") pod \"network-check-target-wxppc\" (UID: \"d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76\") " pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:08:47.220032 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:47.219956 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:08:47.220032 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:47.219976 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:08:47.220032 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:47.219990 2573 projected.go:196] Error preparing data for projected volume kube-api-access-sv77p for pod openshift-network-diagnostics/network-check-target-wxppc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:08:47.220373 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:47.220048 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p podName:d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76 nodeName:}" failed. No retries permitted until 2026-06-11 16:08:49.220028697 +0000 UTC m=+5.054141217 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sv77p" (UniqueName: "kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p") pod "network-check-target-wxppc" (UID: "d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:08:47.221190 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:47.220441 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:08:47.221190 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:47.220507 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret podName:5a412864-1268-4787-be57-d25fd2e7340d nodeName:}" failed. No retries permitted until 2026-06-11 16:08:49.22047931 +0000 UTC m=+5.054591830 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret") pod "global-pull-secret-syncer-tkm6l" (UID: "5a412864-1268-4787-be57-d25fd2e7340d") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:08:47.394890 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:47.394856 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 11 16:08:47.562858 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:47.562722 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:47.563058 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:47.562850 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:08:47.563308 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:47.563279 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:08:47.563413 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:47.563389 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:08:47.563517 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:47.563456 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:08:47.563573 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:47.563524 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:08:47.572850 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:47.572816 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-10 16:03:45 +0000 UTC" deadline="2028-02-21 07:11:23.163565188 +0000 UTC" Jun 11 16:08:47.572850 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:47.572850 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14871h2m35.590720097s" Jun 11 16:08:49.136459 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:49.136423 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs\") pod \"network-metrics-daemon-hnstd\" (UID: \"7cc2d1d8-23c4-462c-854b-c005f8e29363\") " pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:08:49.137014 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:49.136568 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:08:49.137014 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:49.136631 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs podName:7cc2d1d8-23c4-462c-854b-c005f8e29363 nodeName:}" failed. No retries permitted until 2026-06-11 16:08:53.136611806 +0000 UTC m=+8.970724326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs") pod "network-metrics-daemon-hnstd" (UID: "7cc2d1d8-23c4-462c-854b-c005f8e29363") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:08:49.237518 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:49.237481 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret\") pod \"global-pull-secret-syncer-tkm6l\" (UID: \"5a412864-1268-4787-be57-d25fd2e7340d\") " pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:49.237694 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:49.237570 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv77p\" (UniqueName: \"kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p\") pod \"network-check-target-wxppc\" (UID: \"d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76\") " pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:08:49.237753 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:49.237714 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:08:49.237753 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:49.237730 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:08:49.237753 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:49.237743 2573 projected.go:196] Error preparing data for projected volume kube-api-access-sv77p for pod openshift-network-diagnostics/network-check-target-wxppc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:08:49.237906 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:49.237798 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p podName:d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76 nodeName:}" failed. No retries permitted until 2026-06-11 16:08:53.237780415 +0000 UTC m=+9.071892944 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sv77p" (UniqueName: "kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p") pod "network-check-target-wxppc" (UID: "d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:08:49.237906 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:49.237864 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:08:49.237906 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:49.237898 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret podName:5a412864-1268-4787-be57-d25fd2e7340d nodeName:}" failed. No retries permitted until 2026-06-11 16:08:53.23788702 +0000 UTC m=+9.071999541 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret") pod "global-pull-secret-syncer-tkm6l" (UID: "5a412864-1268-4787-be57-d25fd2e7340d") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:08:49.563457 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:49.562734 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:08:49.563457 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:49.562845 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:08:49.563457 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:49.562905 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:49.563457 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:49.562999 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:08:49.563457 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:49.563047 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:08:49.563457 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:49.563121 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:08:51.562766 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:51.562311 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:51.562766 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:51.562683 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:08:51.562766 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:51.562384 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:08:51.566736 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:51.566701 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:08:51.567324 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:51.562364 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:08:51.567484 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:51.567454 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:08:53.171042 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:53.170414 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs\") pod \"network-metrics-daemon-hnstd\" (UID: \"7cc2d1d8-23c4-462c-854b-c005f8e29363\") " pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:08:53.171042 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:53.170611 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:08:53.171042 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:53.170680 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs podName:7cc2d1d8-23c4-462c-854b-c005f8e29363 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:01.170661238 +0000 UTC m=+17.004773758 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs") pod "network-metrics-daemon-hnstd" (UID: "7cc2d1d8-23c4-462c-854b-c005f8e29363") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:08:53.271857 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:53.271519 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv77p\" (UniqueName: \"kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p\") pod \"network-check-target-wxppc\" (UID: \"d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76\") " pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:08:53.271857 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:53.271599 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret\") pod \"global-pull-secret-syncer-tkm6l\" (UID: \"5a412864-1268-4787-be57-d25fd2e7340d\") " pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:53.271857 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:53.271706 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:08:53.271857 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:53.271732 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:08:53.271857 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:53.271747 2573 projected.go:196] Error preparing data for projected volume kube-api-access-sv77p for pod openshift-network-diagnostics/network-check-target-wxppc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:08:53.271857 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:53.271781 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:08:53.271857 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:53.271807 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p podName:d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:01.271789461 +0000 UTC m=+17.105901980 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sv77p" (UniqueName: "kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p") pod "network-check-target-wxppc" (UID: "d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:08:53.271857 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:53.271830 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret podName:5a412864-1268-4787-be57-d25fd2e7340d nodeName:}" failed. No retries permitted until 2026-06-11 16:09:01.271815599 +0000 UTC m=+17.105928134 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret") pod "global-pull-secret-syncer-tkm6l" (UID: "5a412864-1268-4787-be57-d25fd2e7340d") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:08:53.562808 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:53.562726 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:53.562980 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:53.562859 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:08:53.563278 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:53.563244 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:08:53.563402 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:53.563348 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:08:53.563402 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:53.563387 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:08:53.563519 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:53.563434 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:08:55.562584 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:55.562538 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:08:55.562584 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:55.562560 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:08:55.563136 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:55.562537 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:55.563136 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:55.562660 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:08:55.563136 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:55.562762 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:08:55.563136 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:55.562855 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:08:57.563077 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:57.563034 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:08:57.563536 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:57.563036 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:08:57.563536 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:57.563051 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:57.563536 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:57.563193 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:08:57.563536 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:57.563262 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:08:57.563536 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:57.563372 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:08:59.563090 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:59.563062 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:08:59.563453 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:59.563156 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:08:59.563453 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:59.563169 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:08:59.563453 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:59.563239 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:08:59.563453 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:08:59.563250 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:08:59.563453 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:08:59.563343 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:09:01.233524 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:01.233104 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs\") pod \"network-metrics-daemon-hnstd\" (UID: \"7cc2d1d8-23c4-462c-854b-c005f8e29363\") " pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:09:01.233524 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:01.233277 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:09:01.233524 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:01.233346 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs podName:7cc2d1d8-23c4-462c-854b-c005f8e29363 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:17.233324208 +0000 UTC m=+33.067436737 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs") pod "network-metrics-daemon-hnstd" (UID: "7cc2d1d8-23c4-462c-854b-c005f8e29363") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:09:01.333602 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:01.333567 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret\") pod \"global-pull-secret-syncer-tkm6l\" (UID: \"5a412864-1268-4787-be57-d25fd2e7340d\") " pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:09:01.333784 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:01.333635 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv77p\" (UniqueName: \"kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p\") pod \"network-check-target-wxppc\" (UID: \"d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76\") " pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:09:01.333784 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:01.333718 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:09:01.333784 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:01.333769 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:09:01.333784 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:01.333787 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:09:01.333975 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:01.333800 2573 projected.go:196] Error preparing data for projected volume kube-api-access-sv77p for pod openshift-network-diagnostics/network-check-target-wxppc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:09:01.333975 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:01.333805 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret podName:5a412864-1268-4787-be57-d25fd2e7340d nodeName:}" failed. No retries permitted until 2026-06-11 16:09:17.333782385 +0000 UTC m=+33.167894904 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret") pod "global-pull-secret-syncer-tkm6l" (UID: "5a412864-1268-4787-be57-d25fd2e7340d") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:09:01.333975 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:01.333847 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p podName:d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:17.333832967 +0000 UTC m=+33.167945498 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sv77p" (UniqueName: "kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p") pod "network-check-target-wxppc" (UID: "d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:09:01.563742 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:01.563209 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:09:01.563742 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:01.563218 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:09:01.563742 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:01.563334 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:09:01.563742 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:01.563218 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:09:01.563742 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:01.563425 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:09:01.563742 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:01.563503 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:09:03.563234 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:03.563181 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:09:03.563796 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:03.563302 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:09:03.563796 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:03.563349 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:09:03.563796 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:03.563362 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:09:03.563796 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:03.563500 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:09:03.563796 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:03.563595 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:09:05.562791 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:05.562751 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:09:05.562791 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:05.562794 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:09:05.563297 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:05.562794 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:09:05.563297 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:05.562882 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:09:05.563297 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:05.563006 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:09:05.563297 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:05.563085 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:09:06.692989 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:06.692170 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" event={"ID":"beb5dfd5-f33d-43d9-9b23-7c025879f438","Type":"ContainerStarted","Data":"c3d7e419212acc0ea2f718b9435c8ede16b5acd4296476aa23df0263f20d0d4b"} Jun 11 16:09:06.699864 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:06.699644 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-111.ec2.internal" event={"ID":"3154ce17f76996f6fd8c8d37ce0304d4","Type":"ContainerStarted","Data":"c46929e6a1fc2b3850d84ff9d99fcc51bd37fb70718ad670a3d4642a986c43cc"} Jun 11 16:09:06.702825 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:06.702805 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" event={"ID":"7acea643-2978-4893-a6a9-bf88a11b531b","Type":"ContainerStarted","Data":"c7aefe268a11542816bc31ee103d5fb3bd4425707c0f2482d613db02a34fdd94"} Jun 11 16:09:06.716898 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:06.715836 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ds2tm" podStartSLOduration=2.101599832 podStartE2EDuration="22.715817083s" totalImagesPullingTime="20.614217251s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:06.714026987 +0000 UTC m=+22.548139525" watchObservedRunningTime="2026-06-11 16:09:06.715817083 +0000 UTC m=+22.549929623" Jun 11 16:09:07.562515 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.562348 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:09:07.562658 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.562402 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:09:07.562658 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:07.562607 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:09:07.562768 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.562402 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:09:07.562768 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:07.562667 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:09:07.562768 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:07.562743 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:09:07.706106 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.706080 2573 generic.go:356] "Generic (PLEG): container finished" podID="b9669133-a0d2-475c-85c1-7ad84b9a59ba" containerID="e95de295141ec27d2de2ecb2afe8ef1be2fd5f6d28e420513a9ff63ceae0ab2d" exitCode=0 Jun 11 16:09:07.706808 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.706160 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8tpf" event={"ID":"b9669133-a0d2-475c-85c1-7ad84b9a59ba","Type":"ContainerDied","Data":"e95de295141ec27d2de2ecb2afe8ef1be2fd5f6d28e420513a9ff63ceae0ab2d"} Jun 11 16:09:07.707603 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.707582 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k25qj" event={"ID":"85d4cd35-2225-4a31-b023-336e1ccbbdc7","Type":"ContainerStarted","Data":"18d512deb36df012856c80b543533859faf9947b69a497374b82be4a7154684b"} Jun 11 16:09:07.709099 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.709065 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" event={"ID":"c5588305-38f6-4734-8cb8-fd14ec235f72","Type":"ContainerStarted","Data":"509b6c9f29035f0414e1a8dce90adf0d8cb15842c95862650602473bf92bc0d8"} Jun 11 16:09:07.710377 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.710350 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s6bfw" event={"ID":"e9654c3d-7ced-4f3a-8eca-9de0abf92ae3","Type":"ContainerStarted","Data":"1056b67308778cb04512aaf258fd325968263cf9f29d2af85a10633c37b0d067"} Jun 11 16:09:07.711785 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.711765 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gmrpd" event={"ID":"17c6b2e4-8351-4bab-a83a-53b33503505d","Type":"ContainerStarted","Data":"5295f3ece36c4e094d4efea91783c08fbd5b1e8c525973cf0a57f4398c010530"} Jun 11 16:09:07.713129 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.713110 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fwc2s" event={"ID":"5010e752-7999-4b90-bba6-6423eab6f4f2","Type":"ContainerStarted","Data":"d9ed80c9d219b182a5bb75cbc9c2ca22073f9240015b2927bc7b9c546c737dcd"} Jun 11 16:09:07.715737 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.715719 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p7nvz_7acea643-2978-4893-a6a9-bf88a11b531b/ovn-acl-logging/0.log" Jun 11 16:09:07.716065 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.716049 2573 generic.go:356] "Generic (PLEG): container finished" podID="7acea643-2978-4893-a6a9-bf88a11b531b" containerID="142fc4f243bf65f958c62bea9098b3377e6416565060ecbbf559a214391f66b4" exitCode=1 Jun 11 16:09:07.716140 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.716113 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" event={"ID":"7acea643-2978-4893-a6a9-bf88a11b531b","Type":"ContainerStarted","Data":"dd8fce136c9604715667745a1b7c0d5279e81a0e16a3560aa4f5e1656e767d78"} Jun 11 16:09:07.716193 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.716165 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" event={"ID":"7acea643-2978-4893-a6a9-bf88a11b531b","Type":"ContainerStarted","Data":"8196d481bb182661eb04a40e0f088f745bb28f5a27a128804468cbcb78b7182c"} Jun 11 16:09:07.716244 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.716193 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" event={"ID":"7acea643-2978-4893-a6a9-bf88a11b531b","Type":"ContainerStarted","Data":"1ca009030b465b726f676548f1e90b7e87ee667e14e688d0e4dab8817ba3c98e"} Jun 11 16:09:07.716244 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.716234 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" event={"ID":"7acea643-2978-4893-a6a9-bf88a11b531b","Type":"ContainerStarted","Data":"f735ed27496f85ba8d8cbf3a33f01e5761d9c73bceb2073e5dd406619601ad3e"} Jun 11 16:09:07.716327 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.716266 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" event={"ID":"7acea643-2978-4893-a6a9-bf88a11b531b","Type":"ContainerDied","Data":"142fc4f243bf65f958c62bea9098b3377e6416565060ecbbf559a214391f66b4"} Jun 11 16:09:07.717328 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.717310 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s6txn" event={"ID":"764eca77-76b0-4ef2-a59f-ad632d07aeac","Type":"ContainerStarted","Data":"230755b300902c12209c10d958c1cabb858fa978f0ce23bfdc04603e32009d02"} Jun 11 16:09:07.718608 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.718587 2573 generic.go:356] "Generic (PLEG): container finished" podID="3023c529b9a574aede626b35cf1e00da" containerID="95ef120ca620c9b2c0520bcd6491cbad03cde8e4b9d7372f8dff19f6d71be42e" exitCode=0 Jun 11 16:09:07.718708 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.718647 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal" event={"ID":"3023c529b9a574aede626b35cf1e00da","Type":"ContainerDied","Data":"95ef120ca620c9b2c0520bcd6491cbad03cde8e4b9d7372f8dff19f6d71be42e"} Jun 11 16:09:07.763568 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.763500 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-111.ec2.internal" podStartSLOduration=22.763482478 podStartE2EDuration="22.763482478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:06.735203977 +0000 UTC m=+22.569316516" watchObservedRunningTime="2026-06-11 16:09:07.763482478 +0000 UTC m=+23.597595018" Jun 11 16:09:07.825384 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.825337 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-k25qj" podStartSLOduration=2.246097278 podStartE2EDuration="22.82532267s" totalImagesPullingTime="20.579225392s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:07.82512822 +0000 UTC m=+23.659240758" watchObservedRunningTime="2026-06-11 16:09:07.82532267 +0000 UTC m=+23.659435206" Jun 11 16:09:07.889064 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.889017 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fwc2s" podStartSLOduration=2.314917513 podStartE2EDuration="22.889002613s" totalImagesPullingTime="20.5740851s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:07.853169123 +0000 UTC m=+23.687281659" watchObservedRunningTime="2026-06-11 16:09:07.889002613 +0000 UTC m=+23.723115162" Jun 11 16:09:07.920563 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.920478 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gmrpd" podStartSLOduration=3.31188784 podStartE2EDuration="23.920463226s" totalImagesPullingTime="20.608575386s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:07.889281479 +0000 UTC m=+23.723394028" watchObservedRunningTime="2026-06-11 16:09:07.920463226 +0000 UTC m=+23.754575762" Jun 11 16:09:07.971863 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:07.971818 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-s6txn" podStartSLOduration=3.17374171 podStartE2EDuration="23.971803902s" totalImagesPullingTime="20.798062192s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:07.920556287 +0000 UTC m=+23.754668857" watchObservedRunningTime="2026-06-11 16:09:07.971803902 +0000 UTC m=+23.805916454" Jun 11 16:09:08.723851 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:08.723814 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal" event={"ID":"3023c529b9a574aede626b35cf1e00da","Type":"ContainerStarted","Data":"6b2a483fa3ea2a1e3e34d995887fa6bf4c3d8708920c4e5626af33948606dccc"} Jun 11 16:09:08.742710 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:08.742655 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-111.ec2.internal" podStartSLOduration=23.742635519 podStartE2EDuration="23.742635519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:08.742516856 +0000 UTC m=+24.576629394" watchObservedRunningTime="2026-06-11 16:09:08.742635519 +0000 UTC m=+24.576748059" Jun 11 16:09:08.742883 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:08.742775 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-s6bfw" podStartSLOduration=4.147385788 podStartE2EDuration="24.742767588s" totalImagesPullingTime="20.5953818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:07.971489813 +0000 UTC m=+23.805602330" watchObservedRunningTime="2026-06-11 16:09:08.742767588 +0000 UTC m=+24.576880125" Jun 11 16:09:09.350210 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:09.350179 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-s6bfw" Jun 11 16:09:09.350727 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:09.350706 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-s6bfw" Jun 11 16:09:09.562555 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:09.562519 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:09:09.562777 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:09.562536 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:09:09.562777 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:09.562647 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:09:09.562777 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:09.562664 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:09:09.562777 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:09.562758 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:09:09.563036 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:09.562851 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:09:09.730416 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:09.730387 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p7nvz_7acea643-2978-4893-a6a9-bf88a11b531b/ovn-acl-logging/0.log" Jun 11 16:09:09.730896 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:09.730763 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" event={"ID":"7acea643-2978-4893-a6a9-bf88a11b531b","Type":"ContainerStarted","Data":"3dd5cc57636cde3578a512e9e920cd65c60612a62ecd918ca39f81c007ec43d4"} Jun 11 16:09:09.731298 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:09.731161 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-s6bfw" Jun 11 16:09:09.731777 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:09.731761 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-s6bfw" Jun 11 16:09:09.998367 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:09.998339 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 11 16:09:10.571118 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:10.570819 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-11T16:09:09.9983617Z","UUID":"f2a002b4-d8f8-4cfa-9e05-3eefdfc4e37c","Handler":null,"Name":"","Endpoint":""} Jun 11 16:09:10.572498 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:10.572474 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 11 16:09:10.572622 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:10.572511 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 11 16:09:10.734950 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:10.734885 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" event={"ID":"c5588305-38f6-4734-8cb8-fd14ec235f72","Type":"ContainerStarted","Data":"4bab10767075fb9e8efe8244d535215a55889ffd20fb4e292bd1afb0b9e16ea0"} Jun 11 16:09:11.563120 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:11.563076 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:09:11.563297 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:11.563206 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:09:11.563595 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:11.563578 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:09:11.563668 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:11.563656 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:09:11.563732 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:11.563712 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:09:11.563855 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:11.563792 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:09:12.743559 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:12.743307 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p7nvz_7acea643-2978-4893-a6a9-bf88a11b531b/ovn-acl-logging/0.log" Jun 11 16:09:12.744323 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:12.744007 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" event={"ID":"7acea643-2978-4893-a6a9-bf88a11b531b","Type":"ContainerStarted","Data":"0d33acf9c48875a070bfae1611604a7811af9a35e8c382de45dc060bf2660c4d"} Jun 11 16:09:12.745979 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:12.745241 2573 scope.go:117] "RemoveContainer" containerID="142fc4f243bf65f958c62bea9098b3377e6416565060ecbbf559a214391f66b4" Jun 11 16:09:12.749203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:12.749181 2573 generic.go:356] "Generic (PLEG): container finished" podID="b9669133-a0d2-475c-85c1-7ad84b9a59ba" containerID="29ed7579226d023b01858ffdfaec86354f512c17eb1887400984bbe9760081d6" exitCode=0 Jun 11 16:09:12.749299 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:12.749262 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8tpf" event={"ID":"b9669133-a0d2-475c-85c1-7ad84b9a59ba","Type":"ContainerDied","Data":"29ed7579226d023b01858ffdfaec86354f512c17eb1887400984bbe9760081d6"} Jun 11 16:09:12.751485 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:12.751459 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" event={"ID":"c5588305-38f6-4734-8cb8-fd14ec235f72","Type":"ContainerStarted","Data":"60ad25a4b833921e745eee21852cb6a3cc47e8d5ba88bfad7208cba0c91ee67a"} Jun 11 16:09:12.856455 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:12.856409 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8clfn" podStartSLOduration=2.300279315 podStartE2EDuration="27.856392489s" totalImagesPullingTime="25.556113174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:12.856292772 +0000 UTC m=+28.690405310" watchObservedRunningTime="2026-06-11 16:09:12.856392489 +0000 UTC m=+28.690505038" Jun 11 16:09:13.562677 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:13.562643 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:09:13.562912 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:13.562683 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:09:13.562912 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:13.562645 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:09:13.562912 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:13.562758 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:09:13.562912 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:13.562811 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:09:13.562912 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:13.562863 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:09:13.757132 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:13.757105 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p7nvz_7acea643-2978-4893-a6a9-bf88a11b531b/ovn-acl-logging/0.log" Jun 11 16:09:13.757555 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:13.757525 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" event={"ID":"7acea643-2978-4893-a6a9-bf88a11b531b","Type":"ContainerStarted","Data":"8c92f3f238449facaa306af286f29b7ca13ecbf82a0ef3c8c48a6220269a8ab9"} Jun 11 16:09:13.757756 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:13.757739 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:09:13.758197 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:13.758178 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:09:13.758325 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:13.758245 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:09:13.774528 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:13.774505 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:09:13.775294 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:13.775273 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:09:13.814565 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:13.814468 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" podStartSLOduration=9.199728094 podStartE2EDuration="29.814446642s" totalImagesPullingTime="20.614718548s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:13.813413059 +0000 UTC m=+29.647525601" watchObservedRunningTime="2026-06-11 16:09:13.814446642 +0000 UTC m=+29.648559182" Jun 11 16:09:13.967126 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:13.967091 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tkm6l"] Jun 11 16:09:13.967298 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:13.967239 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:09:13.967348 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:13.967325 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:09:13.969631 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:13.969602 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wxppc"] Jun 11 16:09:13.969775 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:13.969717 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:09:13.969839 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:13.969811 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:09:13.990691 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:13.990657 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hnstd"] Jun 11 16:09:13.990838 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:13.990771 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:09:13.990886 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:13.990866 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:09:14.761416 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:14.761381 2573 generic.go:356] "Generic (PLEG): container finished" podID="b9669133-a0d2-475c-85c1-7ad84b9a59ba" containerID="cc3568fd0750506ba377821c8cd26e56eecf0a7bb5a13b905b12039f0b79b4d1" exitCode=0 Jun 11 16:09:14.761858 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:14.761467 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8tpf" event={"ID":"b9669133-a0d2-475c-85c1-7ad84b9a59ba","Type":"ContainerDied","Data":"cc3568fd0750506ba377821c8cd26e56eecf0a7bb5a13b905b12039f0b79b4d1"} Jun 11 16:09:15.563265 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:15.563233 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:09:15.563438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:15.563272 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:09:15.563438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:15.563272 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:09:15.563438 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:15.563337 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:09:15.563438 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:15.563425 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:09:15.563569 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:15.563494 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:09:16.768476 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:16.768443 2573 generic.go:356] "Generic (PLEG): container finished" podID="b9669133-a0d2-475c-85c1-7ad84b9a59ba" containerID="c47a0f2d8f18892a81361831b9343f9e1a63a8ce104c436da3e900d99ab88bc8" exitCode=0 Jun 11 16:09:16.768999 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:16.768492 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8tpf" event={"ID":"b9669133-a0d2-475c-85c1-7ad84b9a59ba","Type":"ContainerDied","Data":"c47a0f2d8f18892a81361831b9343f9e1a63a8ce104c436da3e900d99ab88bc8"} Jun 11 16:09:17.248413 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:17.248376 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs\") pod \"network-metrics-daemon-hnstd\" (UID: \"7cc2d1d8-23c4-462c-854b-c005f8e29363\") " pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:09:17.248603 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:17.248550 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:09:17.248664 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:17.248625 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs podName:7cc2d1d8-23c4-462c-854b-c005f8e29363 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:49.248604598 +0000 UTC m=+65.082717122 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs") pod "network-metrics-daemon-hnstd" (UID: "7cc2d1d8-23c4-462c-854b-c005f8e29363") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 11 16:09:17.349559 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:17.349525 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret\") pod \"global-pull-secret-syncer-tkm6l\" (UID: \"5a412864-1268-4787-be57-d25fd2e7340d\") " pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:09:17.349734 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:17.349592 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv77p\" (UniqueName: \"kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p\") pod \"network-check-target-wxppc\" (UID: \"d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76\") " pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:09:17.349734 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:17.349706 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 11 16:09:17.349734 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:17.349720 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 11 16:09:17.349877 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:17.349737 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 11 16:09:17.349877 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:17.349750 2573 projected.go:196] Error preparing data for projected volume kube-api-access-sv77p for pod openshift-network-diagnostics/network-check-target-wxppc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:09:17.349877 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:17.349777 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret podName:5a412864-1268-4787-be57-d25fd2e7340d nodeName:}" failed. No retries permitted until 2026-06-11 16:09:49.349761061 +0000 UTC m=+65.183873581 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret") pod "global-pull-secret-syncer-tkm6l" (UID: "5a412864-1268-4787-be57-d25fd2e7340d") : object "kube-system"/"original-pull-secret" not registered Jun 11 16:09:17.349877 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:17.349803 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p podName:d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:49.349785135 +0000 UTC m=+65.183897664 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sv77p" (UniqueName: "kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p") pod "network-check-target-wxppc" (UID: "d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 11 16:09:17.562601 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:17.562468 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:09:17.562601 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:17.562505 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:09:17.562601 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:17.562503 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:09:17.562838 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:17.562597 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tkm6l" podUID="5a412864-1268-4787-be57-d25fd2e7340d" Jun 11 16:09:17.562838 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:17.562690 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:09:17.562838 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:17.562750 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wxppc" podUID="d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76" Jun 11 16:09:19.531838 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.531803 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-111.ec2.internal" event="NodeReady" Jun 11 16:09:19.532562 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.531984 2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 11 16:09:19.562264 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.562220 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:09:19.562264 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.562252 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:09:19.562489 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.562254 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:09:19.566167 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.566144 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 16:09:19.566309 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.566196 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 11 16:09:19.566309 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.566147 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2w2xq\"" Jun 11 16:09:19.566434 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.566399 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 16:09:19.566543 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.566464 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 11 16:09:19.567013 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.566973 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kx8mw\"" Jun 11 16:09:19.600692 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.600669 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5c8b9dd97b-q4tcx"] Jun 11 16:09:19.604326 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.604310 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.621419 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.621401 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 11 16:09:19.621538 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.621493 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 11 16:09:19.621538 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.621513 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 11 16:09:19.621653 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.621631 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2gg8w\"" Jun 11 16:09:19.629061 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.629043 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 11 16:09:19.654267 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.654244 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-g8957"] Jun 11 16:09:19.657012 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.656982 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g8957" Jun 11 16:09:19.661147 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.661125 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xwb7k"] Jun 11 16:09:19.663580 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.663562 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c8b9dd97b-q4tcx"] Jun 11 16:09:19.663694 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.663657 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:19.679982 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.679964 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 11 16:09:19.682471 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.682453 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-phqs4\"" Jun 11 16:09:19.683154 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.683136 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 11 16:09:19.683312 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.683290 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pbxmp\"" Jun 11 16:09:19.684277 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.684260 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 11 16:09:19.684383 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.684260 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 11 16:09:19.684383 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.684378 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 11 16:09:19.703557 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.703538 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g8957"] Jun 11 16:09:19.707306 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.707282 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xwb7k"] Jun 11 16:09:19.767178 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.767140 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk944\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-kube-api-access-kk944\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.767337 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.767219 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:19.767337 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.767257 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/593fc31d-18b7-4f07-b658-720932870bf6-config-volume\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:19.767337 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.767273 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.767337 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.767296 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/593fc31d-18b7-4f07-b658-720932870bf6-tmp-dir\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:19.767337 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.767312 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx5h9\" (UniqueName: \"kubernetes.io/projected/593fc31d-18b7-4f07-b658-720932870bf6-kube-api-access-jx5h9\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:19.767337 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.767333 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4de12cb4-2845-45d5-8f17-6850ff64baaa-installation-pull-secrets\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.767561 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.767352 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4de12cb4-2845-45d5-8f17-6850ff64baaa-trusted-ca\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.767561 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.767370 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9p9r\" (UniqueName: \"kubernetes.io/projected/b4ff7d18-6726-42f5-9f39-a36097a40fb2-kube-api-access-h9p9r\") pod \"ingress-canary-g8957\" (UID: \"b4ff7d18-6726-42f5-9f39-a36097a40fb2\") " pod="openshift-ingress-canary/ingress-canary-g8957" Jun 11 16:09:19.767561 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.767392 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4de12cb4-2845-45d5-8f17-6850ff64baaa-image-registry-private-configuration\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.767561 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.767414 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4de12cb4-2845-45d5-8f17-6850ff64baaa-ca-trust-extracted\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.767561 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.767432 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4ff7d18-6726-42f5-9f39-a36097a40fb2-cert\") pod \"ingress-canary-g8957\" (UID: \"b4ff7d18-6726-42f5-9f39-a36097a40fb2\") " pod="openshift-ingress-canary/ingress-canary-g8957" Jun 11 16:09:19.767561 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.767445 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-bound-sa-token\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.767561 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.767459 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-certificates\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.869172 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.868186 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk944\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-kube-api-access-kk944\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.869172 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.868268 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:19.869172 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.868311 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/593fc31d-18b7-4f07-b658-720932870bf6-config-volume\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:19.869172 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.868331 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.869172 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.868363 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/593fc31d-18b7-4f07-b658-720932870bf6-tmp-dir\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:19.869172 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.868386 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx5h9\" (UniqueName: \"kubernetes.io/projected/593fc31d-18b7-4f07-b658-720932870bf6-kube-api-access-jx5h9\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:19.869172 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.868410 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4de12cb4-2845-45d5-8f17-6850ff64baaa-installation-pull-secrets\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.869172 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.868434 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4de12cb4-2845-45d5-8f17-6850ff64baaa-trusted-ca\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.869172 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:19.868449 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:09:19.869172 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.868459 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9p9r\" (UniqueName: \"kubernetes.io/projected/b4ff7d18-6726-42f5-9f39-a36097a40fb2-kube-api-access-h9p9r\") pod \"ingress-canary-g8957\" (UID: \"b4ff7d18-6726-42f5-9f39-a36097a40fb2\") " pod="openshift-ingress-canary/ingress-canary-g8957" Jun 11 16:09:19.869172 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:19.868452 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:09:19.869172 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:19.868529 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls podName:593fc31d-18b7-4f07-b658-720932870bf6 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:20.368504283 +0000 UTC m=+36.202616813 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls") pod "dns-default-xwb7k" (UID: "593fc31d-18b7-4f07-b658-720932870bf6") : secret "dns-default-metrics-tls" not found Jun 11 16:09:19.869172 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.868588 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4de12cb4-2845-45d5-8f17-6850ff64baaa-image-registry-private-configuration\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.869172 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:19.868528 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c8b9dd97b-q4tcx: secret "image-registry-tls" not found Jun 11 16:09:19.869172 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.868620 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4de12cb4-2845-45d5-8f17-6850ff64baaa-ca-trust-extracted\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.869172 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:19.868663 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls podName:4de12cb4-2845-45d5-8f17-6850ff64baaa nodeName:}" failed. No retries permitted until 2026-06-11 16:09:20.368644075 +0000 UTC m=+36.202756596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls") pod "image-registry-5c8b9dd97b-q4tcx" (UID: "4de12cb4-2845-45d5-8f17-6850ff64baaa") : secret "image-registry-tls" not found Jun 11 16:09:19.869172 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.868686 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4ff7d18-6726-42f5-9f39-a36097a40fb2-cert\") pod \"ingress-canary-g8957\" (UID: \"b4ff7d18-6726-42f5-9f39-a36097a40fb2\") " pod="openshift-ingress-canary/ingress-canary-g8957" Jun 11 16:09:19.869971 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.868718 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-bound-sa-token\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.869971 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.868743 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-certificates\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.869971 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.868840 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/593fc31d-18b7-4f07-b658-720932870bf6-tmp-dir\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:19.869971 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.869061 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4de12cb4-2845-45d5-8f17-6850ff64baaa-ca-trust-extracted\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.869971 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.869111 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/593fc31d-18b7-4f07-b658-720932870bf6-config-volume\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:19.869971 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.869342 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-certificates\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.869971 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.869441 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4de12cb4-2845-45d5-8f17-6850ff64baaa-trusted-ca\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.873340 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.873318 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4de12cb4-2845-45d5-8f17-6850ff64baaa-image-registry-private-configuration\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.873466 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.873340 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4de12cb4-2845-45d5-8f17-6850ff64baaa-installation-pull-secrets\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.873466 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.873376 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b4ff7d18-6726-42f5-9f39-a36097a40fb2-cert\") pod \"ingress-canary-g8957\" (UID: \"b4ff7d18-6726-42f5-9f39-a36097a40fb2\") " pod="openshift-ingress-canary/ingress-canary-g8957" Jun 11 16:09:19.889939 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.889892 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx5h9\" (UniqueName: \"kubernetes.io/projected/593fc31d-18b7-4f07-b658-720932870bf6-kube-api-access-jx5h9\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:19.889939 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.889913 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-bound-sa-token\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.890723 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.890694 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk944\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-kube-api-access-kk944\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:19.890829 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.890761 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9p9r\" (UniqueName: \"kubernetes.io/projected/b4ff7d18-6726-42f5-9f39-a36097a40fb2-kube-api-access-h9p9r\") pod \"ingress-canary-g8957\" (UID: \"b4ff7d18-6726-42f5-9f39-a36097a40fb2\") " pod="openshift-ingress-canary/ingress-canary-g8957" Jun 11 16:09:19.967183 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:19.967149 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g8957" Jun 11 16:09:20.372892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:20.372848 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:20.373073 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:20.372980 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:20.373073 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:20.373027 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:09:20.373073 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:20.373051 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c8b9dd97b-q4tcx: secret "image-registry-tls" not found Jun 11 16:09:20.373209 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:20.373094 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:09:20.373209 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:20.373108 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls podName:4de12cb4-2845-45d5-8f17-6850ff64baaa nodeName:}" failed. No retries permitted until 2026-06-11 16:09:21.373094102 +0000 UTC m=+37.207206618 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls") pod "image-registry-5c8b9dd97b-q4tcx" (UID: "4de12cb4-2845-45d5-8f17-6850ff64baaa") : secret "image-registry-tls" not found Jun 11 16:09:20.373209 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:20.373162 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls podName:593fc31d-18b7-4f07-b658-720932870bf6 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:21.373139009 +0000 UTC m=+37.207251531 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls") pod "dns-default-xwb7k" (UID: "593fc31d-18b7-4f07-b658-720932870bf6") : secret "dns-default-metrics-tls" not found Jun 11 16:09:20.847301 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:20.847090 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g8957"] Jun 11 16:09:20.850193 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:09:20.850170 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4ff7d18_6726_42f5_9f39_a36097a40fb2.slice/crio-1b2cb72b7363ba6a913fd391b8745318295b8a5c7ebbbfc733b24424fa7c03b7 WatchSource:0}: Error finding container 1b2cb72b7363ba6a913fd391b8745318295b8a5c7ebbbfc733b24424fa7c03b7: Status 404 returned error can't find the container with id 1b2cb72b7363ba6a913fd391b8745318295b8a5c7ebbbfc733b24424fa7c03b7 Jun 11 16:09:21.380281 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:21.380193 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:21.380432 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:21.380313 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:21.380432 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:21.380368 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:09:21.380432 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:21.380394 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c8b9dd97b-q4tcx: secret "image-registry-tls" not found Jun 11 16:09:21.380432 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:21.380430 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:09:21.380626 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:21.380453 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls podName:4de12cb4-2845-45d5-8f17-6850ff64baaa nodeName:}" failed. No retries permitted until 2026-06-11 16:09:23.380437589 +0000 UTC m=+39.214550106 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls") pod "image-registry-5c8b9dd97b-q4tcx" (UID: "4de12cb4-2845-45d5-8f17-6850ff64baaa") : secret "image-registry-tls" not found Jun 11 16:09:21.380626 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:21.380479 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls podName:593fc31d-18b7-4f07-b658-720932870bf6 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:23.380463444 +0000 UTC m=+39.214575980 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls") pod "dns-default-xwb7k" (UID: "593fc31d-18b7-4f07-b658-720932870bf6") : secret "dns-default-metrics-tls" not found Jun 11 16:09:21.782892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:21.782851 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g8957" event={"ID":"b4ff7d18-6726-42f5-9f39-a36097a40fb2","Type":"ContainerStarted","Data":"1b2cb72b7363ba6a913fd391b8745318295b8a5c7ebbbfc733b24424fa7c03b7"} Jun 11 16:09:21.786006 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:21.785975 2573 generic.go:356] "Generic (PLEG): container finished" podID="b9669133-a0d2-475c-85c1-7ad84b9a59ba" containerID="0ea713e00f8508b29b7658d6b5629ae060fe7b03db6a6c1f5f2bc1e7883fb68e" exitCode=0 Jun 11 16:09:21.786141 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:21.786028 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8tpf" event={"ID":"b9669133-a0d2-475c-85c1-7ad84b9a59ba","Type":"ContainerDied","Data":"0ea713e00f8508b29b7658d6b5629ae060fe7b03db6a6c1f5f2bc1e7883fb68e"} Jun 11 16:09:22.791623 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:22.791541 2573 generic.go:356] "Generic (PLEG): container finished" podID="b9669133-a0d2-475c-85c1-7ad84b9a59ba" containerID="8b4d20705b20f3b9a850e1b8e2b87114fd98dcebce49ac8c0aadcba7b71482a1" exitCode=0 Jun 11 16:09:22.791623 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:22.791591 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8tpf" event={"ID":"b9669133-a0d2-475c-85c1-7ad84b9a59ba","Type":"ContainerDied","Data":"8b4d20705b20f3b9a850e1b8e2b87114fd98dcebce49ac8c0aadcba7b71482a1"} Jun 11 16:09:23.394410 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:23.394228 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:23.394596 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:23.394446 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:23.394596 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:23.394382 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:09:23.394596 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:23.394484 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c8b9dd97b-q4tcx: secret "image-registry-tls" not found Jun 11 16:09:23.394596 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:23.394535 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:09:23.394596 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:23.394536 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls podName:4de12cb4-2845-45d5-8f17-6850ff64baaa nodeName:}" failed. No retries permitted until 2026-06-11 16:09:27.394521889 +0000 UTC m=+43.228634406 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls") pod "image-registry-5c8b9dd97b-q4tcx" (UID: "4de12cb4-2845-45d5-8f17-6850ff64baaa") : secret "image-registry-tls" not found Jun 11 16:09:23.394596 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:23.394579 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls podName:593fc31d-18b7-4f07-b658-720932870bf6 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:27.394568674 +0000 UTC m=+43.228681192 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls") pod "dns-default-xwb7k" (UID: "593fc31d-18b7-4f07-b658-720932870bf6") : secret "dns-default-metrics-tls" not found Jun 11 16:09:23.797523 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:23.797492 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-t8tpf" event={"ID":"b9669133-a0d2-475c-85c1-7ad84b9a59ba","Type":"ContainerStarted","Data":"2e52a857e062ae5d91b9d60d06dc40d01ef4a78a3416e1f20bed8dbba32fb85c"} Jun 11 16:09:23.798912 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:23.798885 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g8957" event={"ID":"b4ff7d18-6726-42f5-9f39-a36097a40fb2","Type":"ContainerStarted","Data":"b2bd9d8a51cafe9dbcb81e6a49b0df6a94e64e188b6d8aad196738db304aed16"} Jun 11 16:09:23.829154 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:23.829101 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-t8tpf" podStartSLOduration=8.031294925 podStartE2EDuration="38.829081772s" totalImagesPullingTime="30.797786847s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:23.826978546 +0000 UTC m=+39.661091082" watchObservedRunningTime="2026-06-11 16:09:23.829081772 +0000 UTC m=+39.663194312" Jun 11 16:09:23.857126 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:23.857086 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-g8957" podStartSLOduration=2.912680885 podStartE2EDuration="4.857071756s" totalImagesPullingTime="1.944390871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:09:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:23.850573981 +0000 UTC m=+39.684686521" watchObservedRunningTime="2026-06-11 16:09:23.857071756 +0000 UTC m=+39.691184295" Jun 11 16:09:27.428171 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:27.428131 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:27.428603 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:27.428181 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:27.428603 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:27.428286 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:09:27.428603 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:27.428297 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c8b9dd97b-q4tcx: secret "image-registry-tls" not found Jun 11 16:09:27.428603 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:27.428299 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:09:27.428603 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:27.428347 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls podName:4de12cb4-2845-45d5-8f17-6850ff64baaa nodeName:}" failed. No retries permitted until 2026-06-11 16:09:35.428333298 +0000 UTC m=+51.262445814 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls") pod "image-registry-5c8b9dd97b-q4tcx" (UID: "4de12cb4-2845-45d5-8f17-6850ff64baaa") : secret "image-registry-tls" not found Jun 11 16:09:27.428603 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:27.428371 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls podName:593fc31d-18b7-4f07-b658-720932870bf6 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:35.428352349 +0000 UTC m=+51.262464878 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls") pod "dns-default-xwb7k" (UID: "593fc31d-18b7-4f07-b658-720932870bf6") : secret "dns-default-metrics-tls" not found Jun 11 16:09:29.646115 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:29.646081 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk"] Jun 11 16:09:29.759322 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:29.759293 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk"] Jun 11 16:09:29.759486 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:29.759402 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" Jun 11 16:09:29.763259 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:29.763227 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 11 16:09:29.763666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:29.763632 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-p5mh9\"" Jun 11 16:09:29.763666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:29.763653 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 11 16:09:29.763838 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:29.763692 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 11 16:09:29.763838 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:29.763637 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 11 16:09:29.846715 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:29.846681 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq54q\" (UniqueName: \"kubernetes.io/projected/9b9fa028-ca49-431a-9d72-952407339ae7-kube-api-access-nq54q\") pod \"managed-serviceaccount-addon-agent-74bf84f9fb-r47vk\" (UID: \"9b9fa028-ca49-431a-9d72-952407339ae7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" Jun 11 16:09:29.846873 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:29.846741 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b9fa028-ca49-431a-9d72-952407339ae7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74bf84f9fb-r47vk\" (UID: \"9b9fa028-ca49-431a-9d72-952407339ae7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" Jun 11 16:09:29.947352 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:29.947301 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq54q\" (UniqueName: \"kubernetes.io/projected/9b9fa028-ca49-431a-9d72-952407339ae7-kube-api-access-nq54q\") pod \"managed-serviceaccount-addon-agent-74bf84f9fb-r47vk\" (UID: \"9b9fa028-ca49-431a-9d72-952407339ae7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" Jun 11 16:09:29.947539 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:29.947401 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b9fa028-ca49-431a-9d72-952407339ae7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74bf84f9fb-r47vk\" (UID: \"9b9fa028-ca49-431a-9d72-952407339ae7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" Jun 11 16:09:29.951724 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:29.951696 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9b9fa028-ca49-431a-9d72-952407339ae7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74bf84f9fb-r47vk\" (UID: \"9b9fa028-ca49-431a-9d72-952407339ae7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" Jun 11 16:09:29.956077 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:29.956054 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq54q\" (UniqueName: \"kubernetes.io/projected/9b9fa028-ca49-431a-9d72-952407339ae7-kube-api-access-nq54q\") pod \"managed-serviceaccount-addon-agent-74bf84f9fb-r47vk\" (UID: \"9b9fa028-ca49-431a-9d72-952407339ae7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" Jun 11 16:09:30.083642 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:30.083601 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" Jun 11 16:09:30.218705 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:30.218636 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk"] Jun 11 16:09:30.222072 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:09:30.222044 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b9fa028_ca49_431a_9d72_952407339ae7.slice/crio-2b2de74b1411a9a1100972601ddc6667aaf58a4fd79bcd31be90897bf17e734c WatchSource:0}: Error finding container 2b2de74b1411a9a1100972601ddc6667aaf58a4fd79bcd31be90897bf17e734c: Status 404 returned error can't find the container with id 2b2de74b1411a9a1100972601ddc6667aaf58a4fd79bcd31be90897bf17e734c Jun 11 16:09:30.816230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:30.816194 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" event={"ID":"9b9fa028-ca49-431a-9d72-952407339ae7","Type":"ContainerStarted","Data":"2b2de74b1411a9a1100972601ddc6667aaf58a4fd79bcd31be90897bf17e734c"} Jun 11 16:09:34.826700 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:34.826660 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" event={"ID":"9b9fa028-ca49-431a-9d72-952407339ae7","Type":"ContainerStarted","Data":"417b0efad4bac718888088e81688705451546548d70e096d9bc2a291b90d5804"} Jun 11 16:09:34.848232 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:34.848179 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" podStartSLOduration=2.108047647 podStartE2EDuration="5.848162711s" totalImagesPullingTime="3.740115064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:09:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:34.847990662 +0000 UTC m=+50.682103200" watchObservedRunningTime="2026-06-11 16:09:34.848162711 +0000 UTC m=+50.682275250" Jun 11 16:09:35.491374 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:35.491332 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:35.491555 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:35.491390 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:35.491555 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:35.491500 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:09:35.491555 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:35.491529 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:09:35.491555 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:35.491541 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c8b9dd97b-q4tcx: secret "image-registry-tls" not found Jun 11 16:09:35.491720 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:35.491576 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls podName:593fc31d-18b7-4f07-b658-720932870bf6 nodeName:}" failed. No retries permitted until 2026-06-11 16:09:51.491555959 +0000 UTC m=+67.325668490 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls") pod "dns-default-xwb7k" (UID: "593fc31d-18b7-4f07-b658-720932870bf6") : secret "dns-default-metrics-tls" not found Jun 11 16:09:35.491720 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:35.491593 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls podName:4de12cb4-2845-45d5-8f17-6850ff64baaa nodeName:}" failed. No retries permitted until 2026-06-11 16:09:51.491585957 +0000 UTC m=+67.325698474 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls") pod "image-registry-5c8b9dd97b-q4tcx" (UID: "4de12cb4-2845-45d5-8f17-6850ff64baaa") : secret "image-registry-tls" not found Jun 11 16:09:45.777945 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:45.777897 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-p7nvz" Jun 11 16:09:49.294023 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:49.293974 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs\") pod \"network-metrics-daemon-hnstd\" (UID: \"7cc2d1d8-23c4-462c-854b-c005f8e29363\") " pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:09:49.296613 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:49.296592 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 11 16:09:49.304624 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:49.304603 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 11 16:09:49.304714 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:49.304703 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs podName:7cc2d1d8-23c4-462c-854b-c005f8e29363 nodeName:}" failed. No retries permitted until 2026-06-11 16:10:53.304685296 +0000 UTC m=+129.138797813 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs") pod "network-metrics-daemon-hnstd" (UID: "7cc2d1d8-23c4-462c-854b-c005f8e29363") : secret "metrics-daemon-secret" not found Jun 11 16:09:49.394674 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:49.394640 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv77p\" (UniqueName: \"kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p\") pod \"network-check-target-wxppc\" (UID: \"d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76\") " pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:09:49.394768 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:49.394707 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret\") pod \"global-pull-secret-syncer-tkm6l\" (UID: \"5a412864-1268-4787-be57-d25fd2e7340d\") " pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:09:49.397375 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:49.397357 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 11 16:09:49.397788 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:49.397773 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 11 16:09:49.407936 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:49.407900 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 11 16:09:49.408099 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:49.408079 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5a412864-1268-4787-be57-d25fd2e7340d-original-pull-secret\") pod \"global-pull-secret-syncer-tkm6l\" (UID: \"5a412864-1268-4787-be57-d25fd2e7340d\") " pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:09:49.418283 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:49.418261 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv77p\" (UniqueName: \"kubernetes.io/projected/d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76-kube-api-access-sv77p\") pod \"network-check-target-wxppc\" (UID: \"d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76\") " pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:09:49.574955 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:49.574846 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tkm6l" Jun 11 16:09:49.582662 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:49.582604 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2w2xq\"" Jun 11 16:09:49.590532 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:49.590506 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:09:49.734536 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:49.734363 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tkm6l"] Jun 11 16:09:49.736997 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:09:49.736971 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a412864_1268_4787_be57_d25fd2e7340d.slice/crio-e7a0d94da5120f461feeff389501fbb487d24f70b30c119f600a26c9af874805 WatchSource:0}: Error finding container e7a0d94da5120f461feeff389501fbb487d24f70b30c119f600a26c9af874805: Status 404 returned error can't find the container with id e7a0d94da5120f461feeff389501fbb487d24f70b30c119f600a26c9af874805 Jun 11 16:09:49.749731 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:49.749700 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wxppc"] Jun 11 16:09:49.753148 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:09:49.753112 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2b9c7c0_483e_4bdd_b5d0_13c5f3fcdd76.slice/crio-9784caaca02e0be7a931ae9166336c77ac7de9ccf02af12359cb63ad9f52eae4 WatchSource:0}: Error finding container 9784caaca02e0be7a931ae9166336c77ac7de9ccf02af12359cb63ad9f52eae4: Status 404 returned error can't find the container with id 9784caaca02e0be7a931ae9166336c77ac7de9ccf02af12359cb63ad9f52eae4 Jun 11 16:09:49.864408 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:49.864305 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wxppc" event={"ID":"d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76","Type":"ContainerStarted","Data":"9784caaca02e0be7a931ae9166336c77ac7de9ccf02af12359cb63ad9f52eae4"} Jun 11 16:09:49.865334 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:49.865315 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tkm6l" event={"ID":"5a412864-1268-4787-be57-d25fd2e7340d","Type":"ContainerStarted","Data":"e7a0d94da5120f461feeff389501fbb487d24f70b30c119f600a26c9af874805"} Jun 11 16:09:51.511217 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:51.511171 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:09:51.511785 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:51.511272 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:09:51.511911 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:51.511883 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:09:51.512003 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:51.511916 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c8b9dd97b-q4tcx: secret "image-registry-tls" not found Jun 11 16:09:51.512049 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:51.512013 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls podName:4de12cb4-2845-45d5-8f17-6850ff64baaa nodeName:}" failed. No retries permitted until 2026-06-11 16:10:23.511991182 +0000 UTC m=+99.346103724 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls") pod "image-registry-5c8b9dd97b-q4tcx" (UID: "4de12cb4-2845-45d5-8f17-6850ff64baaa") : secret "image-registry-tls" not found Jun 11 16:09:51.512097 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:51.512049 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:09:51.512097 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:09:51.512095 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls podName:593fc31d-18b7-4f07-b658-720932870bf6 nodeName:}" failed. No retries permitted until 2026-06-11 16:10:23.512084219 +0000 UTC m=+99.346196740 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls") pod "dns-default-xwb7k" (UID: "593fc31d-18b7-4f07-b658-720932870bf6") : secret "dns-default-metrics-tls" not found Jun 11 16:09:53.878394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:53.878354 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wxppc" event={"ID":"d2b9c7c0-483e-4bdd-b5d0-13c5f3fcdd76","Type":"ContainerStarted","Data":"fea2139cbf7e48923e7c6005ff778c97c596f916ec3f2546809d69c59cd9039a"} Jun 11 16:09:53.878831 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:53.878489 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:09:53.901383 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:53.901329 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wxppc" podStartSLOduration=65.019385151 podStartE2EDuration="1m8.901313377s" totalImagesPullingTime="3.881928226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:53.901179651 +0000 UTC m=+69.735292191" watchObservedRunningTime="2026-06-11 16:09:53.901313377 +0000 UTC m=+69.735425916" Jun 11 16:09:55.884795 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:55.884758 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tkm6l" event={"ID":"5a412864-1268-4787-be57-d25fd2e7340d","Type":"ContainerStarted","Data":"7e4348d7916013d189a9d25d906ce3a8a4af2fa1f9cf9d3b868c9d6464ca0845"} Jun 11 16:09:55.905318 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:09:55.905268 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tkm6l" podStartSLOduration=65.367350898 podStartE2EDuration="1m10.905253387s" totalImagesPullingTime="5.537902489s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:09:55.900775075 +0000 UTC m=+71.734887614" watchObservedRunningTime="2026-06-11 16:09:55.905253387 +0000 UTC m=+71.739365976" Jun 11 16:10:23.543522 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:10:23.543477 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:10:23.543977 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:10:23.543532 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:10:23.543977 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:10:23.543639 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:10:23.543977 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:10:23.543671 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:10:23.543977 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:10:23.543687 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c8b9dd97b-q4tcx: secret "image-registry-tls" not found Jun 11 16:10:23.543977 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:10:23.543716 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls podName:593fc31d-18b7-4f07-b658-720932870bf6 nodeName:}" failed. No retries permitted until 2026-06-11 16:11:27.543700126 +0000 UTC m=+163.377812643 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls") pod "dns-default-xwb7k" (UID: "593fc31d-18b7-4f07-b658-720932870bf6") : secret "dns-default-metrics-tls" not found Jun 11 16:10:23.543977 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:10:23.543768 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls podName:4de12cb4-2845-45d5-8f17-6850ff64baaa nodeName:}" failed. No retries permitted until 2026-06-11 16:11:27.5437499 +0000 UTC m=+163.377862433 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls") pod "image-registry-5c8b9dd97b-q4tcx" (UID: "4de12cb4-2845-45d5-8f17-6850ff64baaa") : secret "image-registry-tls" not found Jun 11 16:10:24.884237 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:10:24.884207 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wxppc" Jun 11 16:10:53.352677 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:10:53.352635 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs\") pod \"network-metrics-daemon-hnstd\" (UID: \"7cc2d1d8-23c4-462c-854b-c005f8e29363\") " pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:10:53.353161 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:10:53.352751 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 11 16:10:53.353161 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:10:53.352805 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs podName:7cc2d1d8-23c4-462c-854b-c005f8e29363 nodeName:}" failed. No retries permitted until 2026-06-11 16:12:55.352790845 +0000 UTC m=+251.186903362 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs") pod "network-metrics-daemon-hnstd" (UID: "7cc2d1d8-23c4-462c-854b-c005f8e29363") : secret "metrics-daemon-secret" not found Jun 11 16:11:22.585234 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:11:22.585193 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:11:22.614355 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:11:22.614321 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" podUID="4de12cb4-2845-45d5-8f17-6850ff64baaa" Jun 11 16:11:22.674036 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:11:22.673999 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-xwb7k" podUID="593fc31d-18b7-4f07-b658-720932870bf6" Jun 11 16:11:23.093808 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:11:23.093781 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xwb7k" Jun 11 16:11:23.093808 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:11:23.093814 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:11:27.604531 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:11:27.604471 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:11:27.605022 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:11:27.604587 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:11:27.605022 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:11:27.604612 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 11 16:11:27.605022 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:11:27.604635 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c8b9dd97b-q4tcx: secret "image-registry-tls" not found Jun 11 16:11:27.605022 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:11:27.604691 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls podName:4de12cb4-2845-45d5-8f17-6850ff64baaa nodeName:}" failed. No retries permitted until 2026-06-11 16:13:29.604676517 +0000 UTC m=+285.438789034 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls") pod "image-registry-5c8b9dd97b-q4tcx" (UID: "4de12cb4-2845-45d5-8f17-6850ff64baaa") : secret "image-registry-tls" not found Jun 11 16:11:27.605022 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:11:27.604696 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 11 16:11:27.605022 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:11:27.604760 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls podName:593fc31d-18b7-4f07-b658-720932870bf6 nodeName:}" failed. No retries permitted until 2026-06-11 16:13:29.604743794 +0000 UTC m=+285.438856314 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls") pod "dns-default-xwb7k" (UID: "593fc31d-18b7-4f07-b658-720932870bf6") : secret "dns-default-metrics-tls" not found Jun 11 16:11:34.564138 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:11:34.564106 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:11:35.124018 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:11:35.123976 2573 generic.go:356] "Generic (PLEG): container finished" podID="9b9fa028-ca49-431a-9d72-952407339ae7" containerID="417b0efad4bac718888088e81688705451546548d70e096d9bc2a291b90d5804" exitCode=255 Jun 11 16:11:35.124216 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:11:35.124041 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" event={"ID":"9b9fa028-ca49-431a-9d72-952407339ae7","Type":"ContainerDied","Data":"417b0efad4bac718888088e81688705451546548d70e096d9bc2a291b90d5804"} Jun 11 16:11:35.129564 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:11:35.129541 2573 scope.go:117] "RemoveContainer" containerID="417b0efad4bac718888088e81688705451546548d70e096d9bc2a291b90d5804" Jun 11 16:11:36.129220 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:11:36.129186 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" event={"ID":"9b9fa028-ca49-431a-9d72-952407339ae7","Type":"ContainerStarted","Data":"e8d5d2daf69e7afe68396dfa86ed311be6eccdf37389751c4be8979115ee401b"} Jun 11 16:12:55.439685 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:55.439630 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs\") pod \"network-metrics-daemon-hnstd\" (UID: \"7cc2d1d8-23c4-462c-854b-c005f8e29363\") " pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:12:55.440224 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:12:55.439783 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 11 16:12:55.440224 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:12:55.439846 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs podName:7cc2d1d8-23c4-462c-854b-c005f8e29363 nodeName:}" failed. No retries permitted until 2026-06-11 16:14:57.439830872 +0000 UTC m=+373.273943388 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs") pod "network-metrics-daemon-hnstd" (UID: "7cc2d1d8-23c4-462c-854b-c005f8e29363") : secret "metrics-daemon-secret" not found Jun 11 16:12:59.299377 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.299345 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-f5469b655-n2xhl"] Jun 11 16:12:59.302123 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.302103 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:12:59.305385 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.305363 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 11 16:12:59.305385 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.305374 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 11 16:12:59.305672 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.305654 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 11 16:12:59.305747 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.305721 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 11 16:12:59.305904 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.305889 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 11 16:12:59.305993 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.305943 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 11 16:12:59.306086 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.306069 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-twq47\"" Jun 11 16:12:59.366596 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.366566 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-stats-auth\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:12:59.366596 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.366596 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pthjw\" (UniqueName: \"kubernetes.io/projected/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-kube-api-access-pthjw\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:12:59.366832 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.366635 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-default-certificate\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:12:59.366832 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.366659 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:12:59.366832 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.366723 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:12:59.388783 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.388747 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-f5469b655-n2xhl"] Jun 11 16:12:59.467692 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.467660 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:12:59.467692 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.467693 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:12:59.467970 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:12:59.467824 2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 16:12:59.467970 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.467833 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-stats-auth\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:12:59.467970 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:12:59.467844 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle podName:67e9fb06-c44d-4bd4-bc05-a4df6a246e9d nodeName:}" failed. No retries permitted until 2026-06-11 16:12:59.96782978 +0000 UTC m=+255.801942331 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle") pod "router-default-f5469b655-n2xhl" (UID: "67e9fb06-c44d-4bd4-bc05-a4df6a246e9d") : configmap references non-existent config key: service-ca.crt Jun 11 16:12:59.467970 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.467868 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pthjw\" (UniqueName: \"kubernetes.io/projected/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-kube-api-access-pthjw\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:12:59.467970 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:12:59.467895 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs podName:67e9fb06-c44d-4bd4-bc05-a4df6a246e9d nodeName:}" failed. No retries permitted until 2026-06-11 16:12:59.967876925 +0000 UTC m=+255.801989451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs") pod "router-default-f5469b655-n2xhl" (UID: "67e9fb06-c44d-4bd4-bc05-a4df6a246e9d") : secret "router-metrics-certs-default" not found Jun 11 16:12:59.468237 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.467994 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-default-certificate\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:12:59.470496 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.470473 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-stats-auth\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:12:59.470600 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.470516 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-default-certificate\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:12:59.488726 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.488704 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pthjw\" (UniqueName: \"kubernetes.io/projected/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-kube-api-access-pthjw\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:12:59.971701 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.971661 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:12:59.971701 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:12:59.971708 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:12:59.972014 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:12:59.971831 2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 16:12:59.972014 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:12:59.971880 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle podName:67e9fb06-c44d-4bd4-bc05-a4df6a246e9d nodeName:}" failed. No retries permitted until 2026-06-11 16:13:00.971863907 +0000 UTC m=+256.805976454 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle") pod "router-default-f5469b655-n2xhl" (UID: "67e9fb06-c44d-4bd4-bc05-a4df6a246e9d") : configmap references non-existent config key: service-ca.crt Jun 11 16:12:59.972014 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:12:59.971908 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs podName:67e9fb06-c44d-4bd4-bc05-a4df6a246e9d nodeName:}" failed. No retries permitted until 2026-06-11 16:13:00.971889676 +0000 UTC m=+256.806002193 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs") pod "router-default-f5469b655-n2xhl" (UID: "67e9fb06-c44d-4bd4-bc05-a4df6a246e9d") : secret "router-metrics-certs-default" not found Jun 11 16:13:00.978721 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:00.978668 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:13:00.979283 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:00.978776 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:13:00.979283 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:00.978857 2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 16:13:00.979283 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:00.978911 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle podName:67e9fb06-c44d-4bd4-bc05-a4df6a246e9d nodeName:}" failed. No retries permitted until 2026-06-11 16:13:02.978883471 +0000 UTC m=+258.812996006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle") pod "router-default-f5469b655-n2xhl" (UID: "67e9fb06-c44d-4bd4-bc05-a4df6a246e9d") : configmap references non-existent config key: service-ca.crt Jun 11 16:13:00.979283 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:00.978969 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs podName:67e9fb06-c44d-4bd4-bc05-a4df6a246e9d nodeName:}" failed. No retries permitted until 2026-06-11 16:13:02.978956149 +0000 UTC m=+258.813068670 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs") pod "router-default-f5469b655-n2xhl" (UID: "67e9fb06-c44d-4bd4-bc05-a4df6a246e9d") : secret "router-metrics-certs-default" not found Jun 11 16:13:02.993565 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:02.993484 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:13:02.993565 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:02.993523 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:13:02.994019 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:02.993632 2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 16:13:02.994019 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:02.993635 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle podName:67e9fb06-c44d-4bd4-bc05-a4df6a246e9d nodeName:}" failed. No retries permitted until 2026-06-11 16:13:06.993620304 +0000 UTC m=+262.827732821 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle") pod "router-default-f5469b655-n2xhl" (UID: "67e9fb06-c44d-4bd4-bc05-a4df6a246e9d") : configmap references non-existent config key: service-ca.crt Jun 11 16:13:02.994019 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:02.993737 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs podName:67e9fb06-c44d-4bd4-bc05-a4df6a246e9d nodeName:}" failed. No retries permitted until 2026-06-11 16:13:06.993722748 +0000 UTC m=+262.827835455 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs") pod "router-default-f5469b655-n2xhl" (UID: "67e9fb06-c44d-4bd4-bc05-a4df6a246e9d") : secret "router-metrics-certs-default" not found Jun 11 16:13:04.672936 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:04.672886 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-4ldr7"] Jun 11 16:13:04.675704 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:04.675687 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-4ldr7" Jun 11 16:13:04.687684 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:04.687642 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-7pxdc\"" Jun 11 16:13:04.687787 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:04.687704 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 11 16:13:04.688254 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:04.688241 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 11 16:13:04.709543 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:04.709512 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-4ldr7"] Jun 11 16:13:04.806615 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:04.806574 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpf7k\" (UniqueName: \"kubernetes.io/projected/f2379c57-3673-4fbd-84a8-936789b26e10-kube-api-access-wpf7k\") pod \"migrator-955ffdf5f-4ldr7\" (UID: \"f2379c57-3673-4fbd-84a8-936789b26e10\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-4ldr7" Jun 11 16:13:04.907475 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:04.907443 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpf7k\" (UniqueName: \"kubernetes.io/projected/f2379c57-3673-4fbd-84a8-936789b26e10-kube-api-access-wpf7k\") pod \"migrator-955ffdf5f-4ldr7\" (UID: \"f2379c57-3673-4fbd-84a8-936789b26e10\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-4ldr7" Jun 11 16:13:04.925475 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:04.925413 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpf7k\" (UniqueName: \"kubernetes.io/projected/f2379c57-3673-4fbd-84a8-936789b26e10-kube-api-access-wpf7k\") pod \"migrator-955ffdf5f-4ldr7\" (UID: \"f2379c57-3673-4fbd-84a8-936789b26e10\") " pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-4ldr7" Jun 11 16:13:04.983793 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:04.983764 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-4ldr7" Jun 11 16:13:05.130441 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:05.130404 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-955ffdf5f-4ldr7"] Jun 11 16:13:05.134040 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:13:05.134009 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2379c57_3673_4fbd_84a8_936789b26e10.slice/crio-0026cefd3ffd3c1dffa5d26b084ce51ea6341476587339d6fbe42fc41a9f35b0 WatchSource:0}: Error finding container 0026cefd3ffd3c1dffa5d26b084ce51ea6341476587339d6fbe42fc41a9f35b0: Status 404 returned error can't find the container with id 0026cefd3ffd3c1dffa5d26b084ce51ea6341476587339d6fbe42fc41a9f35b0 Jun 11 16:13:05.376761 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:05.376673 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-4ldr7" event={"ID":"f2379c57-3673-4fbd-84a8-936789b26e10","Type":"ContainerStarted","Data":"0026cefd3ffd3c1dffa5d26b084ce51ea6341476587339d6fbe42fc41a9f35b0"} Jun 11 16:13:07.025279 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:07.025246 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:13:07.025657 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:07.025288 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:13:07.025657 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:07.025391 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle podName:67e9fb06-c44d-4bd4-bc05-a4df6a246e9d nodeName:}" failed. No retries permitted until 2026-06-11 16:13:15.025377993 +0000 UTC m=+270.859490510 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle") pod "router-default-f5469b655-n2xhl" (UID: "67e9fb06-c44d-4bd4-bc05-a4df6a246e9d") : configmap references non-existent config key: service-ca.crt Jun 11 16:13:07.025657 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:07.025398 2573 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 11 16:13:07.025657 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:07.025456 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs podName:67e9fb06-c44d-4bd4-bc05-a4df6a246e9d nodeName:}" failed. No retries permitted until 2026-06-11 16:13:15.025441506 +0000 UTC m=+270.859554043 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs") pod "router-default-f5469b655-n2xhl" (UID: "67e9fb06-c44d-4bd4-bc05-a4df6a246e9d") : secret "router-metrics-certs-default" not found Jun 11 16:13:07.385037 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:07.384954 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-4ldr7" event={"ID":"f2379c57-3673-4fbd-84a8-936789b26e10","Type":"ContainerStarted","Data":"012239571e13d051bcb1621a26037c8e1dddf388daa7ef579ab309b37a6f7a56"} Jun 11 16:13:07.385037 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:07.385005 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-4ldr7" event={"ID":"f2379c57-3673-4fbd-84a8-936789b26e10","Type":"ContainerStarted","Data":"1cf876ed9f4b956815a7c2b6eb86db6e078919f4079302039fb732245b523d5c"} Jun 11 16:13:07.414714 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:07.414666 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-955ffdf5f-4ldr7" podStartSLOduration=2.209510267 podStartE2EDuration="3.414651212s" totalImagesPullingTime="1.205140945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:13:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:07.41365354 +0000 UTC m=+263.247766079" watchObservedRunningTime="2026-06-11 16:13:07.414651212 +0000 UTC m=+263.248763750" Jun 11 16:13:08.430269 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:08.430242 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k25qj_85d4cd35-2225-4a31-b023-336e1ccbbdc7/dns-node-resolver/0.log" Jun 11 16:13:09.431157 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:09.431128 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gmrpd_17c6b2e4-8351-4bab-a83a-53b33503505d/node-ca/0.log" Jun 11 16:13:10.242715 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:10.242689 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g8957_b4ff7d18-6726-42f5-9f39-a36097a40fb2/serve-healthcheck-canary/0.log" Jun 11 16:13:10.644518 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:10.644444 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-4ldr7_f2379c57-3673-4fbd-84a8-936789b26e10/migrator/0.log" Jun 11 16:13:10.837146 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:10.837120 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-4ldr7_f2379c57-3673-4fbd-84a8-936789b26e10/graceful-termination/0.log" Jun 11 16:13:15.083514 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:15.083473 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:13:15.083514 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:15.083517 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:13:15.084092 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:15.083691 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle podName:67e9fb06-c44d-4bd4-bc05-a4df6a246e9d nodeName:}" failed. No retries permitted until 2026-06-11 16:13:31.083671411 +0000 UTC m=+286.917783944 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle") pod "router-default-f5469b655-n2xhl" (UID: "67e9fb06-c44d-4bd4-bc05-a4df6a246e9d") : configmap references non-existent config key: service-ca.crt Jun 11 16:13:15.086165 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:15.086138 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-metrics-certs\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:13:26.094665 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:26.094625 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-xwb7k" podUID="593fc31d-18b7-4f07-b658-720932870bf6" Jun 11 16:13:26.094665 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:26.094625 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" podUID="4de12cb4-2845-45d5-8f17-6850ff64baaa" Jun 11 16:13:26.438187 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:26.438158 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:13:26.438327 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:26.438163 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xwb7k" Jun 11 16:13:29.692604 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:29.692569 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:13:29.693001 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:29.692613 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:13:29.695084 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:29.695050 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/593fc31d-18b7-4f07-b658-720932870bf6-metrics-tls\") pod \"dns-default-xwb7k\" (UID: \"593fc31d-18b7-4f07-b658-720932870bf6\") " pod="openshift-dns/dns-default-xwb7k" Jun 11 16:13:29.695209 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:29.695123 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls\") pod \"image-registry-5c8b9dd97b-q4tcx\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:13:29.741464 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:29.741435 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pbxmp\"" Jun 11 16:13:29.741464 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:29.741436 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2gg8w\"" Jun 11 16:13:29.749083 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:29.749064 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xwb7k" Jun 11 16:13:29.749171 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:29.749156 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:13:29.889820 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:29.889788 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c8b9dd97b-q4tcx"] Jun 11 16:13:29.892898 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:13:29.892874 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4de12cb4_2845_45d5_8f17_6850ff64baaa.slice/crio-fa01883a2f2aa221a664052d6147bc25925489e48e954d7bda90208c3db2c536 WatchSource:0}: Error finding container fa01883a2f2aa221a664052d6147bc25925489e48e954d7bda90208c3db2c536: Status 404 returned error can't find the container with id fa01883a2f2aa221a664052d6147bc25925489e48e954d7bda90208c3db2c536 Jun 11 16:13:29.901983 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:29.901961 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xwb7k"] Jun 11 16:13:29.905140 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:13:29.905118 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod593fc31d_18b7_4f07_b658_720932870bf6.slice/crio-7241658c818eca791d500e220b66dea23b2fc020a96b605dd1f4540b515be17e WatchSource:0}: Error finding container 7241658c818eca791d500e220b66dea23b2fc020a96b605dd1f4540b515be17e: Status 404 returned error can't find the container with id 7241658c818eca791d500e220b66dea23b2fc020a96b605dd1f4540b515be17e Jun 11 16:13:30.267138 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.267068 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bt9lv"] Jun 11 16:13:30.271457 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.271436 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bt9lv" Jun 11 16:13:30.278158 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.278134 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 11 16:13:30.278708 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.278690 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 11 16:13:30.278823 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.278805 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 11 16:13:30.285565 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.285542 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 11 16:13:30.285898 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.285878 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sksvz\"" Jun 11 16:13:30.296172 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.296145 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bt9lv"] Jun 11 16:13:30.398013 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.397985 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/729d8eef-e25d-44e9-a4e8-df71a5ba6b7d-data-volume\") pod \"insights-runtime-extractor-bt9lv\" (UID: \"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d\") " pod="openshift-insights/insights-runtime-extractor-bt9lv" Jun 11 16:13:30.398183 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.398050 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/729d8eef-e25d-44e9-a4e8-df71a5ba6b7d-crio-socket\") pod \"insights-runtime-extractor-bt9lv\" (UID: \"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d\") " pod="openshift-insights/insights-runtime-extractor-bt9lv" Jun 11 16:13:30.398183 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.398074 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/729d8eef-e25d-44e9-a4e8-df71a5ba6b7d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bt9lv\" (UID: \"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d\") " pod="openshift-insights/insights-runtime-extractor-bt9lv" Jun 11 16:13:30.398183 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.398089 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/729d8eef-e25d-44e9-a4e8-df71a5ba6b7d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bt9lv\" (UID: \"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d\") " pod="openshift-insights/insights-runtime-extractor-bt9lv" Jun 11 16:13:30.398183 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.398107 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5kpq\" (UniqueName: \"kubernetes.io/projected/729d8eef-e25d-44e9-a4e8-df71a5ba6b7d-kube-api-access-j5kpq\") pod \"insights-runtime-extractor-bt9lv\" (UID: \"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d\") " pod="openshift-insights/insights-runtime-extractor-bt9lv" Jun 11 16:13:30.452042 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.452009 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" event={"ID":"4de12cb4-2845-45d5-8f17-6850ff64baaa","Type":"ContainerStarted","Data":"2fedcd6562c61d489ec599f4f72237961d05ebcf2aed00521f63ac682e2b8f63"} Jun 11 16:13:30.452231 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.452067 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" event={"ID":"4de12cb4-2845-45d5-8f17-6850ff64baaa","Type":"ContainerStarted","Data":"fa01883a2f2aa221a664052d6147bc25925489e48e954d7bda90208c3db2c536"} Jun 11 16:13:30.452231 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.452098 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:13:30.453250 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.453219 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xwb7k" event={"ID":"593fc31d-18b7-4f07-b658-720932870bf6","Type":"ContainerStarted","Data":"7241658c818eca791d500e220b66dea23b2fc020a96b605dd1f4540b515be17e"} Jun 11 16:13:30.487541 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.487487 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" podStartSLOduration=285.48746887 podStartE2EDuration="4m45.48746887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:08:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:30.487031199 +0000 UTC m=+286.321143739" watchObservedRunningTime="2026-06-11 16:13:30.48746887 +0000 UTC m=+286.321581410" Jun 11 16:13:30.499198 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.499172 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/729d8eef-e25d-44e9-a4e8-df71a5ba6b7d-data-volume\") pod \"insights-runtime-extractor-bt9lv\" (UID: \"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d\") " pod="openshift-insights/insights-runtime-extractor-bt9lv" Jun 11 16:13:30.499266 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.499223 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/729d8eef-e25d-44e9-a4e8-df71a5ba6b7d-crio-socket\") pod \"insights-runtime-extractor-bt9lv\" (UID: \"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d\") " pod="openshift-insights/insights-runtime-extractor-bt9lv" Jun 11 16:13:30.499266 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.499240 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/729d8eef-e25d-44e9-a4e8-df71a5ba6b7d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bt9lv\" (UID: \"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d\") " pod="openshift-insights/insights-runtime-extractor-bt9lv" Jun 11 16:13:30.499646 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.499320 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/729d8eef-e25d-44e9-a4e8-df71a5ba6b7d-crio-socket\") pod \"insights-runtime-extractor-bt9lv\" (UID: \"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d\") " pod="openshift-insights/insights-runtime-extractor-bt9lv" Jun 11 16:13:30.499646 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.499366 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/729d8eef-e25d-44e9-a4e8-df71a5ba6b7d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bt9lv\" (UID: \"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d\") " pod="openshift-insights/insights-runtime-extractor-bt9lv" Jun 11 16:13:30.499646 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.499398 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5kpq\" (UniqueName: \"kubernetes.io/projected/729d8eef-e25d-44e9-a4e8-df71a5ba6b7d-kube-api-access-j5kpq\") pod \"insights-runtime-extractor-bt9lv\" (UID: \"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d\") " pod="openshift-insights/insights-runtime-extractor-bt9lv" Jun 11 16:13:30.499646 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.499567 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/729d8eef-e25d-44e9-a4e8-df71a5ba6b7d-data-volume\") pod \"insights-runtime-extractor-bt9lv\" (UID: \"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d\") " pod="openshift-insights/insights-runtime-extractor-bt9lv" Jun 11 16:13:30.499845 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.499809 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/729d8eef-e25d-44e9-a4e8-df71a5ba6b7d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bt9lv\" (UID: \"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d\") " pod="openshift-insights/insights-runtime-extractor-bt9lv" Jun 11 16:13:30.502252 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.502234 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/729d8eef-e25d-44e9-a4e8-df71a5ba6b7d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bt9lv\" (UID: \"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d\") " pod="openshift-insights/insights-runtime-extractor-bt9lv" Jun 11 16:13:30.508573 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.508553 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5kpq\" (UniqueName: \"kubernetes.io/projected/729d8eef-e25d-44e9-a4e8-df71a5ba6b7d-kube-api-access-j5kpq\") pod \"insights-runtime-extractor-bt9lv\" (UID: \"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d\") " pod="openshift-insights/insights-runtime-extractor-bt9lv" Jun 11 16:13:30.580494 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.580411 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bt9lv" Jun 11 16:13:30.713897 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:30.713856 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bt9lv"] Jun 11 16:13:30.715954 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:13:30.715912 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod729d8eef_e25d_44e9_a4e8_df71a5ba6b7d.slice/crio-87b6d8f224b216849f26db14c0ae133b2f8995af7c86fa0356244795e9addfa6 WatchSource:0}: Error finding container 87b6d8f224b216849f26db14c0ae133b2f8995af7c86fa0356244795e9addfa6: Status 404 returned error can't find the container with id 87b6d8f224b216849f26db14c0ae133b2f8995af7c86fa0356244795e9addfa6 Jun 11 16:13:31.104394 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:31.104348 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:13:31.105020 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:31.104998 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e9fb06-c44d-4bd4-bc05-a4df6a246e9d-service-ca-bundle\") pod \"router-default-f5469b655-n2xhl\" (UID: \"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d\") " pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:13:31.110297 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:31.110272 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:13:31.243484 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:31.243452 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-f5469b655-n2xhl"] Jun 11 16:13:31.313278 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:13:31.313237 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e9fb06_c44d_4bd4_bc05_a4df6a246e9d.slice/crio-9f7dcde90ee47672e8a457b310e18ee1670943314a0a10963f29943ccf4ea447 WatchSource:0}: Error finding container 9f7dcde90ee47672e8a457b310e18ee1670943314a0a10963f29943ccf4ea447: Status 404 returned error can't find the container with id 9f7dcde90ee47672e8a457b310e18ee1670943314a0a10963f29943ccf4ea447 Jun 11 16:13:31.457842 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:31.457797 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-f5469b655-n2xhl" event={"ID":"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d","Type":"ContainerStarted","Data":"ae63088d8d092770a09420f8daf7a3db8888881d551d1722cfee40174a2fe523"} Jun 11 16:13:31.458021 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:31.457864 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-f5469b655-n2xhl" event={"ID":"67e9fb06-c44d-4bd4-bc05-a4df6a246e9d","Type":"ContainerStarted","Data":"9f7dcde90ee47672e8a457b310e18ee1670943314a0a10963f29943ccf4ea447"} Jun 11 16:13:31.459597 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:31.459572 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bt9lv" event={"ID":"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d","Type":"ContainerStarted","Data":"20055c8d92dea5fcbf02f1dbc3d576bcb0a34a250c1f54d35d2e68996335f9e5"} Jun 11 16:13:31.459734 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:31.459620 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bt9lv" event={"ID":"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d","Type":"ContainerStarted","Data":"e06bc73329804e1043498a6463efb366ca9f5cd69b28b8e764b1c0fc00c8a8a9"} Jun 11 16:13:31.459734 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:31.459647 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bt9lv" event={"ID":"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d","Type":"ContainerStarted","Data":"87b6d8f224b216849f26db14c0ae133b2f8995af7c86fa0356244795e9addfa6"} Jun 11 16:13:31.479426 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:31.479368 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-f5469b655-n2xhl" podStartSLOduration=32.479350812 podStartE2EDuration="32.479350812s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:12:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:31.478201523 +0000 UTC m=+287.312314062" watchObservedRunningTime="2026-06-11 16:13:31.479350812 +0000 UTC m=+287.313463350" Jun 11 16:13:32.111403 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:32.111354 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:13:32.114437 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:32.114408 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:13:32.463471 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:32.463432 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:13:32.464898 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:32.464871 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-f5469b655-n2xhl" Jun 11 16:13:33.468516 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:33.468427 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bt9lv" event={"ID":"729d8eef-e25d-44e9-a4e8-df71a5ba6b7d","Type":"ContainerStarted","Data":"0ac968bb7cc46d824e95c5a38b58ed76fde9697dd7dc43edb85ab5693f6a3f36"} Jun 11 16:13:33.470219 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:33.470196 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xwb7k" event={"ID":"593fc31d-18b7-4f07-b658-720932870bf6","Type":"ContainerStarted","Data":"34705a777c9daa6a5e9b25d1bd53e8b0f2f83eca46984af9975eb1a00276bba4"} Jun 11 16:13:33.470326 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:33.470237 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xwb7k" event={"ID":"593fc31d-18b7-4f07-b658-720932870bf6","Type":"ContainerStarted","Data":"94a506ee1f1ee30ee5ac5f1aa4fd19ea4c5da35794a30cdaea8db81c762d3b32"} Jun 11 16:13:33.470326 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:33.470294 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xwb7k" Jun 11 16:13:33.488112 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:33.488066 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bt9lv" podStartSLOduration=1.113262449 podStartE2EDuration="3.488051575s" totalImagesPullingTime="2.374789126s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:13:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:33.487272728 +0000 UTC m=+289.321385267" watchObservedRunningTime="2026-06-11 16:13:33.488051575 +0000 UTC m=+289.322164138" Jun 11 16:13:33.503960 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:33.503895 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xwb7k" podStartSLOduration=251.210536488 podStartE2EDuration="4m14.503882765s" totalImagesPullingTime="3.293346277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:09:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:33.50294939 +0000 UTC m=+289.337061929" watchObservedRunningTime="2026-06-11 16:13:33.503882765 +0000 UTC m=+289.337995304" Jun 11 16:13:35.478781 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:35.478748 2573 generic.go:356] "Generic (PLEG): container finished" podID="9b9fa028-ca49-431a-9d72-952407339ae7" containerID="e8d5d2daf69e7afe68396dfa86ed311be6eccdf37389751c4be8979115ee401b" exitCode=255 Jun 11 16:13:35.479261 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:35.478827 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" event={"ID":"9b9fa028-ca49-431a-9d72-952407339ae7","Type":"ContainerDied","Data":"e8d5d2daf69e7afe68396dfa86ed311be6eccdf37389751c4be8979115ee401b"} Jun 11 16:13:35.479261 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:35.478892 2573 scope.go:117] "RemoveContainer" containerID="417b0efad4bac718888088e81688705451546548d70e096d9bc2a291b90d5804" Jun 11 16:13:35.485341 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:35.485316 2573 scope.go:117] "RemoveContainer" containerID="e8d5d2daf69e7afe68396dfa86ed311be6eccdf37389751c4be8979115ee401b" Jun 11 16:13:35.485566 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:35.485538 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-74bf84f9fb-r47vk_open-cluster-management-agent-addon(9b9fa028-ca49-431a-9d72-952407339ae7)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" podUID="9b9fa028-ca49-431a-9d72-952407339ae7" Jun 11 16:13:37.565498 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:37.565439 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-hnstd" podUID="7cc2d1d8-23c4-462c-854b-c005f8e29363" Jun 11 16:13:38.176020 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.175984 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs"] Jun 11 16:13:38.180416 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.180399 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" Jun 11 16:13:38.183729 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.183707 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-wstn8\"" Jun 11 16:13:38.183729 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.183723 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 11 16:13:38.183914 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.183743 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 11 16:13:38.183914 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.183711 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 11 16:13:38.183914 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.183711 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 11 16:13:38.183914 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.183706 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 11 16:13:38.197246 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.197228 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs"] Jun 11 16:13:38.361499 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.361461 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7887f863-9af9-45a9-9d02-73da43f4aff2-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-8cnxs\" (UID: \"7887f863-9af9-45a9-9d02-73da43f4aff2\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" Jun 11 16:13:38.361499 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.361505 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l947l\" (UniqueName: \"kubernetes.io/projected/7887f863-9af9-45a9-9d02-73da43f4aff2-kube-api-access-l947l\") pod \"prometheus-operator-7f7d445d84-8cnxs\" (UID: \"7887f863-9af9-45a9-9d02-73da43f4aff2\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" Jun 11 16:13:38.361707 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.361526 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7887f863-9af9-45a9-9d02-73da43f4aff2-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-8cnxs\" (UID: \"7887f863-9af9-45a9-9d02-73da43f4aff2\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" Jun 11 16:13:38.361707 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.361553 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7887f863-9af9-45a9-9d02-73da43f4aff2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-8cnxs\" (UID: \"7887f863-9af9-45a9-9d02-73da43f4aff2\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" Jun 11 16:13:38.461968 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.461862 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7887f863-9af9-45a9-9d02-73da43f4aff2-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-8cnxs\" (UID: \"7887f863-9af9-45a9-9d02-73da43f4aff2\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" Jun 11 16:13:38.461968 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.461903 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l947l\" (UniqueName: \"kubernetes.io/projected/7887f863-9af9-45a9-9d02-73da43f4aff2-kube-api-access-l947l\") pod \"prometheus-operator-7f7d445d84-8cnxs\" (UID: \"7887f863-9af9-45a9-9d02-73da43f4aff2\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" Jun 11 16:13:38.461968 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.461958 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7887f863-9af9-45a9-9d02-73da43f4aff2-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-8cnxs\" (UID: \"7887f863-9af9-45a9-9d02-73da43f4aff2\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" Jun 11 16:13:38.462226 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.462001 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7887f863-9af9-45a9-9d02-73da43f4aff2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-8cnxs\" (UID: \"7887f863-9af9-45a9-9d02-73da43f4aff2\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" Jun 11 16:13:38.462226 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:38.462038 2573 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jun 11 16:13:38.462226 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:38.462116 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7887f863-9af9-45a9-9d02-73da43f4aff2-prometheus-operator-tls podName:7887f863-9af9-45a9-9d02-73da43f4aff2 nodeName:}" failed. No retries permitted until 2026-06-11 16:13:38.962095241 +0000 UTC m=+294.796207771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/7887f863-9af9-45a9-9d02-73da43f4aff2-prometheus-operator-tls") pod "prometheus-operator-7f7d445d84-8cnxs" (UID: "7887f863-9af9-45a9-9d02-73da43f4aff2") : secret "prometheus-operator-tls" not found Jun 11 16:13:38.462750 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.462724 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7887f863-9af9-45a9-9d02-73da43f4aff2-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-8cnxs\" (UID: \"7887f863-9af9-45a9-9d02-73da43f4aff2\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" Jun 11 16:13:38.464535 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.464517 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7887f863-9af9-45a9-9d02-73da43f4aff2-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-8cnxs\" (UID: \"7887f863-9af9-45a9-9d02-73da43f4aff2\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" Jun 11 16:13:38.474659 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.474628 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l947l\" (UniqueName: \"kubernetes.io/projected/7887f863-9af9-45a9-9d02-73da43f4aff2-kube-api-access-l947l\") pod \"prometheus-operator-7f7d445d84-8cnxs\" (UID: \"7887f863-9af9-45a9-9d02-73da43f4aff2\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" Jun 11 16:13:38.965793 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.965753 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7887f863-9af9-45a9-9d02-73da43f4aff2-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-8cnxs\" (UID: \"7887f863-9af9-45a9-9d02-73da43f4aff2\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" Jun 11 16:13:38.968294 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:38.968267 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7887f863-9af9-45a9-9d02-73da43f4aff2-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-8cnxs\" (UID: \"7887f863-9af9-45a9-9d02-73da43f4aff2\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" Jun 11 16:13:39.095122 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:39.095077 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" Jun 11 16:13:39.225216 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:39.225192 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs"] Jun 11 16:13:39.227487 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:13:39.227464 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7887f863_9af9_45a9_9d02_73da43f4aff2.slice/crio-b1e30ee397ed2def287834821c95add210e0b77149c9845b85acc9fc75dbe7c0 WatchSource:0}: Error finding container b1e30ee397ed2def287834821c95add210e0b77149c9845b85acc9fc75dbe7c0: Status 404 returned error can't find the container with id b1e30ee397ed2def287834821c95add210e0b77149c9845b85acc9fc75dbe7c0 Jun 11 16:13:39.493737 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:39.493650 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" event={"ID":"7887f863-9af9-45a9-9d02-73da43f4aff2","Type":"ContainerStarted","Data":"b1e30ee397ed2def287834821c95add210e0b77149c9845b85acc9fc75dbe7c0"} Jun 11 16:13:40.084587 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:40.084544 2573 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" Jun 11 16:13:40.092316 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:40.092286 2573 scope.go:117] "RemoveContainer" containerID="e8d5d2daf69e7afe68396dfa86ed311be6eccdf37389751c4be8979115ee401b" Jun 11 16:13:40.092514 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:40.092489 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-74bf84f9fb-r47vk_open-cluster-management-agent-addon(9b9fa028-ca49-431a-9d72-952407339ae7)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" podUID="9b9fa028-ca49-431a-9d72-952407339ae7" Jun 11 16:13:41.501465 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:41.501431 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" event={"ID":"7887f863-9af9-45a9-9d02-73da43f4aff2","Type":"ContainerStarted","Data":"80b6a8b4accc4730f2fa7ae730dbe643ae723360f4b620628191c6ebe9d2fe62"} Jun 11 16:13:41.501861 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:41.501482 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" event={"ID":"7887f863-9af9-45a9-9d02-73da43f4aff2","Type":"ContainerStarted","Data":"8cc4da46ecb42cf45e1ed3002d96bd70dd6e2755655554b11f05f7677b8ffa19"} Jun 11 16:13:41.521476 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:41.521390 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7f7d445d84-8cnxs" podStartSLOduration=2.142340145 podStartE2EDuration="3.521373448s" totalImagesPullingTime="1.379033303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:13:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:41.520710155 +0000 UTC m=+297.354822694" watchObservedRunningTime="2026-06-11 16:13:41.521373448 +0000 UTC m=+297.355485999" Jun 11 16:13:43.476049 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.476021 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xwb7k" Jun 11 16:13:43.570696 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.570652 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8"] Jun 11 16:13:43.573936 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.573902 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.576351 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.576329 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jun 11 16:13:43.576498 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.576481 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jun 11 16:13:43.576760 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.576743 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jun 11 16:13:43.578137 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.578120 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-jddzf\"" Jun 11 16:13:43.589152 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.589131 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-f4t65"] Jun 11 16:13:43.595530 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.592470 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8"] Jun 11 16:13:43.595530 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.592481 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.595780 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.595656 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 11 16:13:43.596737 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.595873 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 11 16:13:43.596737 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.595958 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 11 16:13:43.596737 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.596031 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xfzx2\"" Jun 11 16:13:43.701628 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.701597 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/927f7de8-cf3a-48da-9597-bb088f8ee605-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.701628 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.701631 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnppj\" (UniqueName: \"kubernetes.io/projected/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-kube-api-access-hnppj\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.701819 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.701648 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/927f7de8-cf3a-48da-9597-bb088f8ee605-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.701819 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.701677 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/927f7de8-cf3a-48da-9597-bb088f8ee605-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.701819 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.701759 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.701819 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.701788 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-sys\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.701819 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.701813 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/927f7de8-cf3a-48da-9597-bb088f8ee605-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.701991 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.701834 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/927f7de8-cf3a-48da-9597-bb088f8ee605-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.701991 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.701877 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-metrics-client-ca\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.701991 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.701896 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-textfile\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.701991 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.701915 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-accelerators-collector-config\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.701991 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.701967 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-wtmp\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.702135 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.701994 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-root\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.702135 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.702031 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt89p\" (UniqueName: \"kubernetes.io/projected/927f7de8-cf3a-48da-9597-bb088f8ee605-kube-api-access-xt89p\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.702135 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.702053 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-tls\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.803105 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803020 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/927f7de8-cf3a-48da-9597-bb088f8ee605-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.803105 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803064 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnppj\" (UniqueName: \"kubernetes.io/projected/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-kube-api-access-hnppj\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.803105 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803082 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/927f7de8-cf3a-48da-9597-bb088f8ee605-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.803105 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803102 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/927f7de8-cf3a-48da-9597-bb088f8ee605-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.803432 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803141 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.803432 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803158 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-sys\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.803432 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803184 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/927f7de8-cf3a-48da-9597-bb088f8ee605-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.803432 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803210 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/927f7de8-cf3a-48da-9597-bb088f8ee605-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.803432 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803243 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-metrics-client-ca\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.803432 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803266 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-sys\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.803432 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803273 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-textfile\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.803432 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:43.803332 2573 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jun 11 16:13:43.803432 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803345 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-accelerators-collector-config\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.803432 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803386 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-wtmp\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.803432 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:43.803412 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/927f7de8-cf3a-48da-9597-bb088f8ee605-kube-state-metrics-tls podName:927f7de8-cf3a-48da-9597-bb088f8ee605 nodeName:}" failed. No retries permitted until 2026-06-11 16:13:44.303391605 +0000 UTC m=+300.137504142 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/927f7de8-cf3a-48da-9597-bb088f8ee605-kube-state-metrics-tls") pod "kube-state-metrics-57bbf8bfb5-wk9p8" (UID: "927f7de8-cf3a-48da-9597-bb088f8ee605") : secret "kube-state-metrics-tls" not found Jun 11 16:13:43.804094 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803452 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-root\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.804094 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803486 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt89p\" (UniqueName: \"kubernetes.io/projected/927f7de8-cf3a-48da-9597-bb088f8ee605-kube-api-access-xt89p\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.804094 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803519 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-tls\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.804094 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803521 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-wtmp\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.804094 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803610 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-textfile\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.804094 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803826 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-root\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.804094 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.803990 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-accelerators-collector-config\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.804094 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:43.804076 2573 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 11 16:13:43.804390 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.804097 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/927f7de8-cf3a-48da-9597-bb088f8ee605-metrics-client-ca\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.804390 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:43.804140 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-tls podName:4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2 nodeName:}" failed. No retries permitted until 2026-06-11 16:13:44.304124653 +0000 UTC m=+300.138237170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-tls") pod "node-exporter-f4t65" (UID: "4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2") : secret "node-exporter-tls" not found Jun 11 16:13:43.804390 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.804209 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/927f7de8-cf3a-48da-9597-bb088f8ee605-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.804390 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.804330 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-metrics-client-ca\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.804390 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.804354 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/927f7de8-cf3a-48da-9597-bb088f8ee605-volume-directive-shadow\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.805688 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.805660 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.805800 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.805720 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/927f7de8-cf3a-48da-9597-bb088f8ee605-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.813821 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.813799 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnppj\" (UniqueName: \"kubernetes.io/projected/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-kube-api-access-hnppj\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:43.814043 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.814025 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt89p\" (UniqueName: \"kubernetes.io/projected/927f7de8-cf3a-48da-9597-bb088f8ee605-kube-api-access-xt89p\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:43.919763 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.919736 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6657875f97-9s44r"] Jun 11 16:13:43.924067 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.924052 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-9s44r" Jun 11 16:13:43.926155 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.926133 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 11 16:13:43.926285 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.926235 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-9fbt2\"" Jun 11 16:13:43.926404 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.926387 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 11 16:13:43.973997 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:43.973965 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-9s44r"] Jun 11 16:13:44.105506 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.105434 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pv5w\" (UniqueName: \"kubernetes.io/projected/f99600e4-7f1f-4814-b30b-3a82984c46c6-kube-api-access-7pv5w\") pod \"downloads-6657875f97-9s44r\" (UID: \"f99600e4-7f1f-4814-b30b-3a82984c46c6\") " pod="openshift-console/downloads-6657875f97-9s44r" Jun 11 16:13:44.206098 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.206066 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pv5w\" (UniqueName: \"kubernetes.io/projected/f99600e4-7f1f-4814-b30b-3a82984c46c6-kube-api-access-7pv5w\") pod \"downloads-6657875f97-9s44r\" (UID: \"f99600e4-7f1f-4814-b30b-3a82984c46c6\") " pod="openshift-console/downloads-6657875f97-9s44r" Jun 11 16:13:44.215710 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.215687 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pv5w\" (UniqueName: \"kubernetes.io/projected/f99600e4-7f1f-4814-b30b-3a82984c46c6-kube-api-access-7pv5w\") pod \"downloads-6657875f97-9s44r\" (UID: \"f99600e4-7f1f-4814-b30b-3a82984c46c6\") " pod="openshift-console/downloads-6657875f97-9s44r" Jun 11 16:13:44.233487 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.233462 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6657875f97-9s44r" Jun 11 16:13:44.311947 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.310343 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-tls\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:44.311947 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.310438 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/927f7de8-cf3a-48da-9597-bb088f8ee605-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:44.315409 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.315335 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/927f7de8-cf3a-48da-9597-bb088f8ee605-kube-state-metrics-tls\") pod \"kube-state-metrics-57bbf8bfb5-wk9p8\" (UID: \"927f7de8-cf3a-48da-9597-bb088f8ee605\") " pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:44.317366 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.317314 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2-node-exporter-tls\") pod \"node-exporter-f4t65\" (UID: \"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2\") " pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:44.377883 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.377859 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6657875f97-9s44r"] Jun 11 16:13:44.380179 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:13:44.380154 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf99600e4_7f1f_4814_b30b_3a82984c46c6.slice/crio-8079001cdba53c6f763f6b89cebf3a43915efbef7b1daf48a016567ac2c5b5b4 WatchSource:0}: Error finding container 8079001cdba53c6f763f6b89cebf3a43915efbef7b1daf48a016567ac2c5b5b4: Status 404 returned error can't find the container with id 8079001cdba53c6f763f6b89cebf3a43915efbef7b1daf48a016567ac2c5b5b4 Jun 11 16:13:44.504464 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.504435 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-jddzf\"" Jun 11 16:13:44.509113 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.509086 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xfzx2\"" Jun 11 16:13:44.512347 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.512269 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" Jun 11 16:13:44.513278 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.513250 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-9s44r" event={"ID":"f99600e4-7f1f-4814-b30b-3a82984c46c6","Type":"ContainerStarted","Data":"8079001cdba53c6f763f6b89cebf3a43915efbef7b1daf48a016567ac2c5b5b4"} Jun 11 16:13:44.517080 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.517060 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f4t65" Jun 11 16:13:44.517195 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.517151 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p7nvz_7acea643-2978-4893-a6a9-bf88a11b531b/ovn-acl-logging/0.log" Jun 11 16:13:44.517705 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.517684 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p7nvz_7acea643-2978-4893-a6a9-bf88a11b531b/ovn-acl-logging/0.log" Jun 11 16:13:44.522272 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.522254 2573 kubelet.go:1648] "Image garbage collection succeeded" Jun 11 16:13:44.529044 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:13:44.529022 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b2e3c1f_e706_4991_bbc8_eb09cbc3ddd2.slice/crio-6d9b34c3b20cc9ee88312b8d2a6e17b28c02dafb236ded7d931e112ee58a900f WatchSource:0}: Error finding container 6d9b34c3b20cc9ee88312b8d2a6e17b28c02dafb236ded7d931e112ee58a900f: Status 404 returned error can't find the container with id 6d9b34c3b20cc9ee88312b8d2a6e17b28c02dafb236ded7d931e112ee58a900f Jun 11 16:13:44.665350 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:44.665318 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8"] Jun 11 16:13:44.667703 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:13:44.667677 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod927f7de8_cf3a_48da_9597_bb088f8ee605.slice/crio-a80927bb04dab06e20aeb499e177b5e570f8b9677c9ab7e83e16ac36144c3c76 WatchSource:0}: Error finding container a80927bb04dab06e20aeb499e177b5e570f8b9677c9ab7e83e16ac36144c3c76: Status 404 returned error can't find the container with id a80927bb04dab06e20aeb499e177b5e570f8b9677c9ab7e83e16ac36144c3c76 Jun 11 16:13:45.518996 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.518954 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" event={"ID":"927f7de8-cf3a-48da-9597-bb088f8ee605","Type":"ContainerStarted","Data":"a80927bb04dab06e20aeb499e177b5e570f8b9677c9ab7e83e16ac36144c3c76"} Jun 11 16:13:45.521079 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.521050 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f4t65" event={"ID":"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2","Type":"ContainerStarted","Data":"612c2038d99586f4239b5d243b5d7a04a708d5c86bb7b57d90d7c31848e1602d"} Jun 11 16:13:45.521204 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.521106 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f4t65" event={"ID":"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2","Type":"ContainerStarted","Data":"6d9b34c3b20cc9ee88312b8d2a6e17b28c02dafb236ded7d931e112ee58a900f"} Jun 11 16:13:45.556610 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.556583 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd"] Jun 11 16:13:45.560515 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.560496 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.563432 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.563407 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 11 16:13:45.563734 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.563713 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 11 16:13:45.563799 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.563749 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 11 16:13:45.563799 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.563780 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-gwbqr\"" Jun 11 16:13:45.564035 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.564017 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 11 16:13:45.564106 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.564017 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-16l0ii2e0ahmi\"" Jun 11 16:13:45.568173 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.568156 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 11 16:13:45.579978 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.576765 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd"] Jun 11 16:13:45.618429 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.618399 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-thanos-querier-tls\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.618578 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.618481 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-grpc-tls\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.618578 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.618512 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krltn\" (UniqueName: \"kubernetes.io/projected/3d778107-2764-41eb-ba2f-e1042dcc3c5a-kube-api-access-krltn\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.618683 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.618590 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.618738 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.618694 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.618787 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.618739 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d778107-2764-41eb-ba2f-e1042dcc3c5a-metrics-client-ca\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.618906 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.618809 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.619030 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.618960 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.720232 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.720180 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-thanos-querier-tls\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.720425 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.720259 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-grpc-tls\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.720425 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.720292 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krltn\" (UniqueName: \"kubernetes.io/projected/3d778107-2764-41eb-ba2f-e1042dcc3c5a-kube-api-access-krltn\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.720425 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.720334 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.720425 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.720372 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.720425 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.720401 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d778107-2764-41eb-ba2f-e1042dcc3c5a-metrics-client-ca\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.720695 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.720433 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.720695 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.720474 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.721485 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.721454 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d778107-2764-41eb-ba2f-e1042dcc3c5a-metrics-client-ca\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.723560 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.723438 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.723652 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.723611 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-thanos-querier-tls\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.723720 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.723695 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.723820 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.723740 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-grpc-tls\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.723820 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.723776 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.724424 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.724381 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/3d778107-2764-41eb-ba2f-e1042dcc3c5a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.729577 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.729556 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krltn\" (UniqueName: \"kubernetes.io/projected/3d778107-2764-41eb-ba2f-e1042dcc3c5a-kube-api-access-krltn\") pod \"thanos-querier-5ff6dfb9c5-lr8wd\" (UID: \"3d778107-2764-41eb-ba2f-e1042dcc3c5a\") " pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:45.905235 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:45.905195 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:46.064708 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:46.064604 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd"] Jun 11 16:13:46.358833 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:13:46.358770 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d778107_2764_41eb_ba2f_e1042dcc3c5a.slice/crio-15a58c1af7c2cca2f31a33b9a026b0d6d54b66f3e7480440ea0afc572e39e35e WatchSource:0}: Error finding container 15a58c1af7c2cca2f31a33b9a026b0d6d54b66f3e7480440ea0afc572e39e35e: Status 404 returned error can't find the container with id 15a58c1af7c2cca2f31a33b9a026b0d6d54b66f3e7480440ea0afc572e39e35e Jun 11 16:13:46.360984 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:46.360964 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 16:13:46.528248 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:46.527689 2573 generic.go:356] "Generic (PLEG): container finished" podID="4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2" containerID="612c2038d99586f4239b5d243b5d7a04a708d5c86bb7b57d90d7c31848e1602d" exitCode=0 Jun 11 16:13:46.528248 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:46.527830 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f4t65" event={"ID":"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2","Type":"ContainerDied","Data":"612c2038d99586f4239b5d243b5d7a04a708d5c86bb7b57d90d7c31848e1602d"} Jun 11 16:13:46.530227 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:46.530202 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" event={"ID":"927f7de8-cf3a-48da-9597-bb088f8ee605","Type":"ContainerStarted","Data":"5d0d2841a9712b5d3f3575e5a67eed90d8c5fc6ccf2b77c83a3d49ce3c584bf2"} Jun 11 16:13:46.531577 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:46.531555 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" event={"ID":"3d778107-2764-41eb-ba2f-e1042dcc3c5a","Type":"ContainerStarted","Data":"15a58c1af7c2cca2f31a33b9a026b0d6d54b66f3e7480440ea0afc572e39e35e"} Jun 11 16:13:47.538520 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:47.538480 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" event={"ID":"927f7de8-cf3a-48da-9597-bb088f8ee605","Type":"ContainerStarted","Data":"ba656c232db72ddafad8fa9d6c243a6cb30c3656eb8f7118539f4554e1bffeac"} Jun 11 16:13:47.539131 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:47.538556 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" event={"ID":"927f7de8-cf3a-48da-9597-bb088f8ee605","Type":"ContainerStarted","Data":"85cfad2149f614fe0d368536839e0f622829201b8e40a7ed5f67cba3093bd430"} Jun 11 16:13:47.541381 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:47.541353 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f4t65" event={"ID":"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2","Type":"ContainerStarted","Data":"d0ac2643f17eb7a48648b65d51eed127b3d380ae0f8031863840fe98307b149c"} Jun 11 16:13:47.541519 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:47.541411 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f4t65" event={"ID":"4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2","Type":"ContainerStarted","Data":"54dd169e1e9a1b2f2ebe1320bc2d7a8d7acd9cc14e1acd1ea7dc6caff32225d7"} Jun 11 16:13:47.558548 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:47.558499 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-57bbf8bfb5-wk9p8" podStartSLOduration=2.822801727 podStartE2EDuration="4.558483122s" totalImagesPullingTime="1.735681395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:13:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:47.556743066 +0000 UTC m=+303.390855802" watchObservedRunningTime="2026-06-11 16:13:47.558483122 +0000 UTC m=+303.392595660" Jun 11 16:13:48.346404 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:48.346342 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-f4t65" podStartSLOduration=4.409024728 podStartE2EDuration="5.346319511s" totalImagesPullingTime="854.492249ms" totalInitContainerRuntime="82.802534ms" isStatefulPod=true podCreationTimestamp="2026-06-11 16:13:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:47.58217408 +0000 UTC m=+303.416286620" watchObservedRunningTime="2026-06-11 16:13:48.346319511 +0000 UTC m=+304.180432053" Jun 11 16:13:48.346606 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:48.346548 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-47k7t"] Jun 11 16:13:48.351225 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:48.351201 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-47k7t" Jun 11 16:13:48.356440 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:48.356414 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 11 16:13:48.356536 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:48.356455 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-hl64x\"" Jun 11 16:13:48.368662 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:48.368633 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-47k7t"] Jun 11 16:13:48.444295 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:48.444264 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f797967f-8904-4f56-98bd-92dfb9eb5143-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-47k7t\" (UID: \"f797967f-8904-4f56-98bd-92dfb9eb5143\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-47k7t" Jun 11 16:13:48.444295 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:48.444307 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6l8f\" (UniqueName: \"kubernetes.io/projected/f797967f-8904-4f56-98bd-92dfb9eb5143-kube-api-access-j6l8f\") pod \"monitoring-plugin-76b68cc874-47k7t\" (UID: \"f797967f-8904-4f56-98bd-92dfb9eb5143\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-47k7t" Jun 11 16:13:48.544721 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:48.544684 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f797967f-8904-4f56-98bd-92dfb9eb5143-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-47k7t\" (UID: \"f797967f-8904-4f56-98bd-92dfb9eb5143\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-47k7t" Jun 11 16:13:48.545327 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:48.544743 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6l8f\" (UniqueName: \"kubernetes.io/projected/f797967f-8904-4f56-98bd-92dfb9eb5143-kube-api-access-j6l8f\") pod \"monitoring-plugin-76b68cc874-47k7t\" (UID: \"f797967f-8904-4f56-98bd-92dfb9eb5143\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-47k7t" Jun 11 16:13:48.545327 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:48.544839 2573 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 11 16:13:48.545327 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:13:48.544916 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f797967f-8904-4f56-98bd-92dfb9eb5143-monitoring-plugin-cert podName:f797967f-8904-4f56-98bd-92dfb9eb5143 nodeName:}" failed. No retries permitted until 2026-06-11 16:13:49.0448956 +0000 UTC m=+304.879008131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/f797967f-8904-4f56-98bd-92dfb9eb5143-monitoring-plugin-cert") pod "monitoring-plugin-76b68cc874-47k7t" (UID: "f797967f-8904-4f56-98bd-92dfb9eb5143") : secret "monitoring-plugin-cert" not found Jun 11 16:13:48.559019 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:48.558978 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6l8f\" (UniqueName: \"kubernetes.io/projected/f797967f-8904-4f56-98bd-92dfb9eb5143-kube-api-access-j6l8f\") pod \"monitoring-plugin-76b68cc874-47k7t\" (UID: \"f797967f-8904-4f56-98bd-92dfb9eb5143\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-47k7t" Jun 11 16:13:49.048967 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:49.048878 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f797967f-8904-4f56-98bd-92dfb9eb5143-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-47k7t\" (UID: \"f797967f-8904-4f56-98bd-92dfb9eb5143\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-47k7t" Jun 11 16:13:49.051479 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:49.051461 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f797967f-8904-4f56-98bd-92dfb9eb5143-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-47k7t\" (UID: \"f797967f-8904-4f56-98bd-92dfb9eb5143\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-47k7t" Jun 11 16:13:49.262885 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:49.262851 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-47k7t" Jun 11 16:13:49.421200 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:49.421169 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-47k7t"] Jun 11 16:13:49.424965 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:13:49.424912 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf797967f_8904_4f56_98bd_92dfb9eb5143.slice/crio-c00081d7258f9b11a3161ba4b367c2e2810267cb40ebce490de252bc21adf4c2 WatchSource:0}: Error finding container c00081d7258f9b11a3161ba4b367c2e2810267cb40ebce490de252bc21adf4c2: Status 404 returned error can't find the container with id c00081d7258f9b11a3161ba4b367c2e2810267cb40ebce490de252bc21adf4c2 Jun 11 16:13:49.555803 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:49.555764 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" event={"ID":"3d778107-2764-41eb-ba2f-e1042dcc3c5a","Type":"ContainerStarted","Data":"d29dab037028f809684ce1da22aafb911b54cd1f6f53833c563dd69d245bd8b4"} Jun 11 16:13:49.556418 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:49.555856 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" event={"ID":"3d778107-2764-41eb-ba2f-e1042dcc3c5a","Type":"ContainerStarted","Data":"4717af69322779dfa80955489480fc41a71f0254259960ba06142c465f232295"} Jun 11 16:13:49.556418 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:49.555903 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" event={"ID":"3d778107-2764-41eb-ba2f-e1042dcc3c5a","Type":"ContainerStarted","Data":"036edd6397a119e5ae5d8ac18691d9f24e69a4a27e2ddd8c1c59a0e8a649404a"} Jun 11 16:13:49.561249 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:49.561208 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-47k7t" event={"ID":"f797967f-8904-4f56-98bd-92dfb9eb5143","Type":"ContainerStarted","Data":"c00081d7258f9b11a3161ba4b367c2e2810267cb40ebce490de252bc21adf4c2"} Jun 11 16:13:49.563939 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:49.562613 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:13:49.756954 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:49.756872 2573 patch_prober.go:28] interesting pod/image-registry-5c8b9dd97b-q4tcx container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 11 16:13:49.757574 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:49.757255 2573 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" podUID="4de12cb4-2845-45d5-8f17-6850ff64baaa" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 16:13:50.570275 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:50.570238 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" event={"ID":"3d778107-2764-41eb-ba2f-e1042dcc3c5a","Type":"ContainerStarted","Data":"54ced3fadad90be3f491a5ea0bdf688d9b34333f1724f3cb96055f8c5dcd3079"} Jun 11 16:13:50.570729 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:50.570314 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" event={"ID":"3d778107-2764-41eb-ba2f-e1042dcc3c5a","Type":"ContainerStarted","Data":"d5e53d4f21c37a47f6f37c1d9225efadf967c08b8664f1ea77967bb9e8c9cab1"} Jun 11 16:13:50.570729 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:50.570355 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" event={"ID":"3d778107-2764-41eb-ba2f-e1042dcc3c5a","Type":"ContainerStarted","Data":"b2d8895ea69df04ba2a7799676de04d0bc6a11b6b4b66914968c42abe73e017e"} Jun 11 16:13:50.570729 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:50.570394 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:50.599134 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:50.598998 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" podStartSLOduration=2.3510155409999998 podStartE2EDuration="5.598973137s" totalImagesPullingTime="3.247957596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:13:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:50.5975182 +0000 UTC m=+306.431630753" watchObservedRunningTime="2026-06-11 16:13:50.598973137 +0000 UTC m=+306.433085657" Jun 11 16:13:51.464818 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:51.464792 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:13:51.575860 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:51.575822 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-47k7t" event={"ID":"f797967f-8904-4f56-98bd-92dfb9eb5143","Type":"ContainerStarted","Data":"56bd39a4c38585f4de0447440b9914441be9f246f30be8b173d9ae57cec0d933"} Jun 11 16:13:51.576312 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:51.576019 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-47k7t" Jun 11 16:13:51.580488 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:51.580464 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-47k7t" Jun 11 16:13:51.595720 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:51.595675 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-76b68cc874-47k7t" podStartSLOduration=1.622222271 podStartE2EDuration="3.595659607s" totalImagesPullingTime="1.973437336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:13:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:13:51.594701728 +0000 UTC m=+307.428814271" watchObservedRunningTime="2026-06-11 16:13:51.595659607 +0000 UTC m=+307.429772145" Jun 11 16:13:52.950797 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:52.950766 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5c8b9dd97b-q4tcx"] Jun 11 16:13:55.570780 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:55.570754 2573 scope.go:117] "RemoveContainer" containerID="e8d5d2daf69e7afe68396dfa86ed311be6eccdf37389751c4be8979115ee401b" Jun 11 16:13:56.583193 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:56.583076 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5ff6dfb9c5-lr8wd" Jun 11 16:13:56.597686 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:13:56.597656 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bf84f9fb-r47vk" event={"ID":"9b9fa028-ca49-431a-9d72-952407339ae7","Type":"ContainerStarted","Data":"3e24bcdc88b8b50190a0dfc082fdf3ab1563e90e21bb0b3708277b0dc00b2209"} Jun 11 16:14:04.631057 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:04.631004 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6657875f97-9s44r" event={"ID":"f99600e4-7f1f-4814-b30b-3a82984c46c6","Type":"ContainerStarted","Data":"f4d8f1ea677ebfbde7876a26e52f8b62eae8a700b483f6abb57216da0ea840ab"} Jun 11 16:14:04.631507 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:04.631231 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6657875f97-9s44r" Jun 11 16:14:04.632732 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:04.632708 2573 patch_prober.go:28] interesting pod/downloads-6657875f97-9s44r container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.15:8080/\": dial tcp 10.132.0.15:8080: connect: connection refused" start-of-body= Jun 11 16:14:04.632841 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:04.632753 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-6657875f97-9s44r" podUID="f99600e4-7f1f-4814-b30b-3a82984c46c6" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.15:8080/\": dial tcp 10.132.0.15:8080: connect: connection refused" Jun 11 16:14:04.660432 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:04.660366 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-6657875f97-9s44r" podStartSLOduration=1.515381472 podStartE2EDuration="21.660341041s" totalImagesPullingTime="20.144959569s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:13:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:14:04.659026594 +0000 UTC m=+320.493139134" watchObservedRunningTime="2026-06-11 16:14:04.660341041 +0000 UTC m=+320.494453582" Jun 11 16:14:05.655283 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:05.655244 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6657875f97-9s44r" Jun 11 16:14:17.974376 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:17.974310 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" podUID="4de12cb4-2845-45d5-8f17-6850ff64baaa" containerName="registry" containerID="cri-o://2fedcd6562c61d489ec599f4f72237961d05ebcf2aed00521f63ac682e2b8f63" gracePeriod=30 Jun 11 16:14:18.234248 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.234193 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:14:18.326340 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.326307 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls\") pod \"4de12cb4-2845-45d5-8f17-6850ff64baaa\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " Jun 11 16:14:18.326508 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.326358 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-bound-sa-token\") pod \"4de12cb4-2845-45d5-8f17-6850ff64baaa\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " Jun 11 16:14:18.326508 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.326384 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-certificates\") pod \"4de12cb4-2845-45d5-8f17-6850ff64baaa\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " Jun 11 16:14:18.326508 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.326409 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4de12cb4-2845-45d5-8f17-6850ff64baaa-trusted-ca\") pod \"4de12cb4-2845-45d5-8f17-6850ff64baaa\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " Jun 11 16:14:18.326508 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.326438 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4de12cb4-2845-45d5-8f17-6850ff64baaa-installation-pull-secrets\") pod \"4de12cb4-2845-45d5-8f17-6850ff64baaa\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " Jun 11 16:14:18.326508 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.326467 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kk944\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-kube-api-access-kk944\") pod \"4de12cb4-2845-45d5-8f17-6850ff64baaa\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " Jun 11 16:14:18.326508 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.326488 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4de12cb4-2845-45d5-8f17-6850ff64baaa-ca-trust-extracted\") pod \"4de12cb4-2845-45d5-8f17-6850ff64baaa\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " Jun 11 16:14:18.326813 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.326535 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4de12cb4-2845-45d5-8f17-6850ff64baaa-image-registry-private-configuration\") pod \"4de12cb4-2845-45d5-8f17-6850ff64baaa\" (UID: \"4de12cb4-2845-45d5-8f17-6850ff64baaa\") " Jun 11 16:14:18.326964 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.326910 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4de12cb4-2845-45d5-8f17-6850ff64baaa-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4de12cb4-2845-45d5-8f17-6850ff64baaa" (UID: "4de12cb4-2845-45d5-8f17-6850ff64baaa"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:14:18.327062 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.326953 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4de12cb4-2845-45d5-8f17-6850ff64baaa" (UID: "4de12cb4-2845-45d5-8f17-6850ff64baaa"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:14:18.329059 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.329022 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4de12cb4-2845-45d5-8f17-6850ff64baaa" (UID: "4de12cb4-2845-45d5-8f17-6850ff64baaa"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:14:18.329059 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.329021 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4de12cb4-2845-45d5-8f17-6850ff64baaa" (UID: "4de12cb4-2845-45d5-8f17-6850ff64baaa"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:14:18.329583 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.329550 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4de12cb4-2845-45d5-8f17-6850ff64baaa-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4de12cb4-2845-45d5-8f17-6850ff64baaa" (UID: "4de12cb4-2845-45d5-8f17-6850ff64baaa"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:14:18.329674 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.329595 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4de12cb4-2845-45d5-8f17-6850ff64baaa-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4de12cb4-2845-45d5-8f17-6850ff64baaa" (UID: "4de12cb4-2845-45d5-8f17-6850ff64baaa"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 11 16:14:18.329754 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.329738 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-kube-api-access-kk944" (OuterVolumeSpecName: "kube-api-access-kk944") pod "4de12cb4-2845-45d5-8f17-6850ff64baaa" (UID: "4de12cb4-2845-45d5-8f17-6850ff64baaa"). InnerVolumeSpecName "kube-api-access-kk944". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:14:18.335883 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.335858 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4de12cb4-2845-45d5-8f17-6850ff64baaa-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4de12cb4-2845-45d5-8f17-6850ff64baaa" (UID: "4de12cb4-2845-45d5-8f17-6850ff64baaa"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:14:18.427415 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.427385 2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4de12cb4-2845-45d5-8f17-6850ff64baaa-installation-pull-secrets\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:14:18.427415 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.427411 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kk944\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-kube-api-access-kk944\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:14:18.427415 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.427420 2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4de12cb4-2845-45d5-8f17-6850ff64baaa-ca-trust-extracted\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:14:18.427621 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.427429 2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4de12cb4-2845-45d5-8f17-6850ff64baaa-image-registry-private-configuration\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:14:18.427621 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.427439 2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-tls\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:14:18.427621 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.427447 2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4de12cb4-2845-45d5-8f17-6850ff64baaa-bound-sa-token\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:14:18.427621 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.427455 2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4de12cb4-2845-45d5-8f17-6850ff64baaa-registry-certificates\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:14:18.427621 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.427463 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4de12cb4-2845-45d5-8f17-6850ff64baaa-trusted-ca\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:14:18.687690 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.687651 2573 generic.go:356] "Generic (PLEG): container finished" podID="4de12cb4-2845-45d5-8f17-6850ff64baaa" containerID="2fedcd6562c61d489ec599f4f72237961d05ebcf2aed00521f63ac682e2b8f63" exitCode=0 Jun 11 16:14:18.687914 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.687702 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" Jun 11 16:14:18.687914 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.687737 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" event={"ID":"4de12cb4-2845-45d5-8f17-6850ff64baaa","Type":"ContainerDied","Data":"2fedcd6562c61d489ec599f4f72237961d05ebcf2aed00521f63ac682e2b8f63"} Jun 11 16:14:18.687914 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.687786 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c8b9dd97b-q4tcx" event={"ID":"4de12cb4-2845-45d5-8f17-6850ff64baaa","Type":"ContainerDied","Data":"fa01883a2f2aa221a664052d6147bc25925489e48e954d7bda90208c3db2c536"} Jun 11 16:14:18.687914 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.687807 2573 scope.go:117] "RemoveContainer" containerID="2fedcd6562c61d489ec599f4f72237961d05ebcf2aed00521f63ac682e2b8f63" Jun 11 16:14:18.698065 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.698047 2573 scope.go:117] "RemoveContainer" containerID="2fedcd6562c61d489ec599f4f72237961d05ebcf2aed00521f63ac682e2b8f63" Jun 11 16:14:18.698368 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:14:18.698338 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fedcd6562c61d489ec599f4f72237961d05ebcf2aed00521f63ac682e2b8f63\": container with ID starting with 2fedcd6562c61d489ec599f4f72237961d05ebcf2aed00521f63ac682e2b8f63 not found: ID does not exist" containerID="2fedcd6562c61d489ec599f4f72237961d05ebcf2aed00521f63ac682e2b8f63" Jun 11 16:14:18.698428 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.698367 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fedcd6562c61d489ec599f4f72237961d05ebcf2aed00521f63ac682e2b8f63"} err="failed to get container status \"2fedcd6562c61d489ec599f4f72237961d05ebcf2aed00521f63ac682e2b8f63\": rpc error: code = NotFound desc = could not find container \"2fedcd6562c61d489ec599f4f72237961d05ebcf2aed00521f63ac682e2b8f63\": container with ID starting with 2fedcd6562c61d489ec599f4f72237961d05ebcf2aed00521f63ac682e2b8f63 not found: ID does not exist" Jun 11 16:14:18.716840 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.716814 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5c8b9dd97b-q4tcx"] Jun 11 16:14:18.720682 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:18.720662 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5c8b9dd97b-q4tcx"] Jun 11 16:14:20.567257 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:20.567226 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4de12cb4-2845-45d5-8f17-6850ff64baaa" path="/var/lib/kubelet/pods/4de12cb4-2845-45d5-8f17-6850ff64baaa/volumes" Jun 11 16:14:57.448209 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:57.448148 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs\") pod \"network-metrics-daemon-hnstd\" (UID: \"7cc2d1d8-23c4-462c-854b-c005f8e29363\") " pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:14:57.450569 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:57.450542 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cc2d1d8-23c4-462c-854b-c005f8e29363-metrics-certs\") pod \"network-metrics-daemon-hnstd\" (UID: \"7cc2d1d8-23c4-462c-854b-c005f8e29363\") " pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:14:57.666818 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:57.666787 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kx8mw\"" Jun 11 16:14:57.674120 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:57.674100 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hnstd" Jun 11 16:14:57.804718 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:57.804683 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hnstd"] Jun 11 16:14:57.807961 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:14:57.807912 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc2d1d8_23c4_462c_854b_c005f8e29363.slice/crio-f9373ff2c19a440cf48ed1fa8d091c5f3b7f4aee5ba4577b158aa3a0dbae26f2 WatchSource:0}: Error finding container f9373ff2c19a440cf48ed1fa8d091c5f3b7f4aee5ba4577b158aa3a0dbae26f2: Status 404 returned error can't find the container with id f9373ff2c19a440cf48ed1fa8d091c5f3b7f4aee5ba4577b158aa3a0dbae26f2 Jun 11 16:14:57.828974 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:57.828945 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hnstd" event={"ID":"7cc2d1d8-23c4-462c-854b-c005f8e29363","Type":"ContainerStarted","Data":"f9373ff2c19a440cf48ed1fa8d091c5f3b7f4aee5ba4577b158aa3a0dbae26f2"} Jun 11 16:14:59.839752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:59.839714 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hnstd" event={"ID":"7cc2d1d8-23c4-462c-854b-c005f8e29363","Type":"ContainerStarted","Data":"fefb97494a9965a3075f1ae20a49adda0b41582e7521532ff6ff62dd6a301434"} Jun 11 16:14:59.840261 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:59.839771 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hnstd" event={"ID":"7cc2d1d8-23c4-462c-854b-c005f8e29363","Type":"ContainerStarted","Data":"aac5279be462a338bbddb8b53fc233c56c8981cc03b97ab960f67beaa6c39d47"} Jun 11 16:14:59.858622 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:14:59.858576 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hnstd" podStartSLOduration=374.919580842 podStartE2EDuration="6m15.858560995s" totalImagesPullingTime="938.980153ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:14:59.857185192 +0000 UTC m=+375.691297729" watchObservedRunningTime="2026-06-11 16:14:59.858560995 +0000 UTC m=+375.692673542" Jun 11 16:17:25.594607 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.594568 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h"] Jun 11 16:17:25.595097 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.594848 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4de12cb4-2845-45d5-8f17-6850ff64baaa" containerName="registry" Jun 11 16:17:25.595097 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.594859 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4de12cb4-2845-45d5-8f17-6850ff64baaa" containerName="registry" Jun 11 16:17:25.595097 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.594907 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4de12cb4-2845-45d5-8f17-6850ff64baaa" containerName="registry" Jun 11 16:17:25.597790 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.597774 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" Jun 11 16:17:25.600001 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.599978 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 11 16:17:25.600116 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.600041 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 11 16:17:25.600492 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.600477 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qh6np\"" Jun 11 16:17:25.607348 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.607324 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h"] Jun 11 16:17:25.646322 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.646288 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a719c4ed-adbe-44f4-b01a-42f877213941-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h\" (UID: \"a719c4ed-adbe-44f4-b01a-42f877213941\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" Jun 11 16:17:25.646471 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.646340 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb5td\" (UniqueName: \"kubernetes.io/projected/a719c4ed-adbe-44f4-b01a-42f877213941-kube-api-access-gb5td\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h\" (UID: \"a719c4ed-adbe-44f4-b01a-42f877213941\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" Jun 11 16:17:25.646471 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.646409 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a719c4ed-adbe-44f4-b01a-42f877213941-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h\" (UID: \"a719c4ed-adbe-44f4-b01a-42f877213941\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" Jun 11 16:17:25.747645 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.747616 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb5td\" (UniqueName: \"kubernetes.io/projected/a719c4ed-adbe-44f4-b01a-42f877213941-kube-api-access-gb5td\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h\" (UID: \"a719c4ed-adbe-44f4-b01a-42f877213941\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" Jun 11 16:17:25.747798 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.747652 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a719c4ed-adbe-44f4-b01a-42f877213941-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h\" (UID: \"a719c4ed-adbe-44f4-b01a-42f877213941\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" Jun 11 16:17:25.747798 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.747776 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a719c4ed-adbe-44f4-b01a-42f877213941-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h\" (UID: \"a719c4ed-adbe-44f4-b01a-42f877213941\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" Jun 11 16:17:25.748031 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.748012 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a719c4ed-adbe-44f4-b01a-42f877213941-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h\" (UID: \"a719c4ed-adbe-44f4-b01a-42f877213941\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" Jun 11 16:17:25.748081 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.748072 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a719c4ed-adbe-44f4-b01a-42f877213941-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h\" (UID: \"a719c4ed-adbe-44f4-b01a-42f877213941\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" Jun 11 16:17:25.755646 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.755623 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb5td\" (UniqueName: \"kubernetes.io/projected/a719c4ed-adbe-44f4-b01a-42f877213941-kube-api-access-gb5td\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h\" (UID: \"a719c4ed-adbe-44f4-b01a-42f877213941\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" Jun 11 16:17:25.906722 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:25.906693 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" Jun 11 16:17:26.039597 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:26.039564 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h"] Jun 11 16:17:26.042420 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:17:26.042393 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda719c4ed_adbe_44f4_b01a_42f877213941.slice/crio-8f420e0d017d7e1581e006edd114e9d01c4fe87b5520b90f5d97e29fd04cd124 WatchSource:0}: Error finding container 8f420e0d017d7e1581e006edd114e9d01c4fe87b5520b90f5d97e29fd04cd124: Status 404 returned error can't find the container with id 8f420e0d017d7e1581e006edd114e9d01c4fe87b5520b90f5d97e29fd04cd124 Jun 11 16:17:26.370237 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:26.370152 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" event={"ID":"a719c4ed-adbe-44f4-b01a-42f877213941","Type":"ContainerStarted","Data":"8f420e0d017d7e1581e006edd114e9d01c4fe87b5520b90f5d97e29fd04cd124"} Jun 11 16:17:34.406271 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:34.406236 2573 generic.go:356] "Generic (PLEG): container finished" podID="a719c4ed-adbe-44f4-b01a-42f877213941" containerID="d193d09c956269dea241214b4a4174372bccdbe548f6b3288d9043a670a8b3cf" exitCode=0 Jun 11 16:17:34.406648 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:34.406294 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" event={"ID":"a719c4ed-adbe-44f4-b01a-42f877213941","Type":"ContainerDied","Data":"d193d09c956269dea241214b4a4174372bccdbe548f6b3288d9043a670a8b3cf"} Jun 11 16:17:35.411612 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:35.411577 2573 generic.go:356] "Generic (PLEG): container finished" podID="a719c4ed-adbe-44f4-b01a-42f877213941" containerID="d3e0add075c05a41b0fbdfc205902a3ca08a881fadb2c18f8b259d9d65285c29" exitCode=0 Jun 11 16:17:35.411989 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:35.411642 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" event={"ID":"a719c4ed-adbe-44f4-b01a-42f877213941","Type":"ContainerDied","Data":"d3e0add075c05a41b0fbdfc205902a3ca08a881fadb2c18f8b259d9d65285c29"} Jun 11 16:17:44.448714 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:44.448679 2573 generic.go:356] "Generic (PLEG): container finished" podID="a719c4ed-adbe-44f4-b01a-42f877213941" containerID="35ea533e2441e70b329507a7e77bd52df21af0ccb5ceabf6313b62eef1508560" exitCode=0 Jun 11 16:17:44.449168 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:44.448770 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" event={"ID":"a719c4ed-adbe-44f4-b01a-42f877213941","Type":"ContainerDied","Data":"35ea533e2441e70b329507a7e77bd52df21af0ccb5ceabf6313b62eef1508560"} Jun 11 16:17:45.576655 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:45.576632 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" Jun 11 16:17:45.715182 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:45.715103 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a719c4ed-adbe-44f4-b01a-42f877213941-util\") pod \"a719c4ed-adbe-44f4-b01a-42f877213941\" (UID: \"a719c4ed-adbe-44f4-b01a-42f877213941\") " Jun 11 16:17:45.715182 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:45.715158 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gb5td\" (UniqueName: \"kubernetes.io/projected/a719c4ed-adbe-44f4-b01a-42f877213941-kube-api-access-gb5td\") pod \"a719c4ed-adbe-44f4-b01a-42f877213941\" (UID: \"a719c4ed-adbe-44f4-b01a-42f877213941\") " Jun 11 16:17:45.715364 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:45.715208 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a719c4ed-adbe-44f4-b01a-42f877213941-bundle\") pod \"a719c4ed-adbe-44f4-b01a-42f877213941\" (UID: \"a719c4ed-adbe-44f4-b01a-42f877213941\") " Jun 11 16:17:45.716048 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:45.716020 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a719c4ed-adbe-44f4-b01a-42f877213941-bundle" (OuterVolumeSpecName: "bundle") pod "a719c4ed-adbe-44f4-b01a-42f877213941" (UID: "a719c4ed-adbe-44f4-b01a-42f877213941"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:17:45.717484 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:45.717464 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a719c4ed-adbe-44f4-b01a-42f877213941-kube-api-access-gb5td" (OuterVolumeSpecName: "kube-api-access-gb5td") pod "a719c4ed-adbe-44f4-b01a-42f877213941" (UID: "a719c4ed-adbe-44f4-b01a-42f877213941"). InnerVolumeSpecName "kube-api-access-gb5td". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:17:45.722554 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:45.722529 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a719c4ed-adbe-44f4-b01a-42f877213941-util" (OuterVolumeSpecName: "util") pod "a719c4ed-adbe-44f4-b01a-42f877213941" (UID: "a719c4ed-adbe-44f4-b01a-42f877213941"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:17:45.816633 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:45.816593 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gb5td\" (UniqueName: \"kubernetes.io/projected/a719c4ed-adbe-44f4-b01a-42f877213941-kube-api-access-gb5td\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:17:45.816633 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:45.816633 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a719c4ed-adbe-44f4-b01a-42f877213941-bundle\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:17:45.816633 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:45.816645 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a719c4ed-adbe-44f4-b01a-42f877213941-util\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:17:46.457438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:46.457407 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" event={"ID":"a719c4ed-adbe-44f4-b01a-42f877213941","Type":"ContainerDied","Data":"8f420e0d017d7e1581e006edd114e9d01c4fe87b5520b90f5d97e29fd04cd124"} Jun 11 16:17:46.457438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:46.457431 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d68t8h" Jun 11 16:17:46.457689 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:46.457452 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f420e0d017d7e1581e006edd114e9d01c4fe87b5520b90f5d97e29fd04cd124" Jun 11 16:17:53.759246 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.759217 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-rdr79"] Jun 11 16:17:53.759716 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.759527 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a719c4ed-adbe-44f4-b01a-42f877213941" containerName="pull" Jun 11 16:17:53.759716 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.759543 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a719c4ed-adbe-44f4-b01a-42f877213941" containerName="pull" Jun 11 16:17:53.759716 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.759551 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a719c4ed-adbe-44f4-b01a-42f877213941" containerName="extract" Jun 11 16:17:53.759716 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.759556 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a719c4ed-adbe-44f4-b01a-42f877213941" containerName="extract" Jun 11 16:17:53.759716 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.759573 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a719c4ed-adbe-44f4-b01a-42f877213941" containerName="util" Jun 11 16:17:53.759716 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.759578 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a719c4ed-adbe-44f4-b01a-42f877213941" containerName="util" Jun 11 16:17:53.759716 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.759626 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a719c4ed-adbe-44f4-b01a-42f877213941" containerName="extract" Jun 11 16:17:53.763695 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.763668 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-rdr79" Jun 11 16:17:53.766860 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.766825 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-zcvm8\"" Jun 11 16:17:53.767024 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.766881 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 11 16:17:53.767024 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.766948 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 11 16:17:53.779139 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.779115 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-rdr79"] Jun 11 16:17:53.875627 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.875593 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpfnk\" (UniqueName: \"kubernetes.io/projected/7be9c8f7-6bd2-4377-b06a-be5995d31d04-kube-api-access-bpfnk\") pod \"cert-manager-operator-controller-manager-57757f7cfc-rdr79\" (UID: \"7be9c8f7-6bd2-4377-b06a-be5995d31d04\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-rdr79" Jun 11 16:17:53.875792 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.875633 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7be9c8f7-6bd2-4377-b06a-be5995d31d04-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-rdr79\" (UID: \"7be9c8f7-6bd2-4377-b06a-be5995d31d04\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-rdr79" Jun 11 16:17:53.976866 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.976835 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpfnk\" (UniqueName: \"kubernetes.io/projected/7be9c8f7-6bd2-4377-b06a-be5995d31d04-kube-api-access-bpfnk\") pod \"cert-manager-operator-controller-manager-57757f7cfc-rdr79\" (UID: \"7be9c8f7-6bd2-4377-b06a-be5995d31d04\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-rdr79" Jun 11 16:17:53.977042 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.976872 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7be9c8f7-6bd2-4377-b06a-be5995d31d04-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-rdr79\" (UID: \"7be9c8f7-6bd2-4377-b06a-be5995d31d04\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-rdr79" Jun 11 16:17:53.977238 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.977223 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7be9c8f7-6bd2-4377-b06a-be5995d31d04-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-rdr79\" (UID: \"7be9c8f7-6bd2-4377-b06a-be5995d31d04\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-rdr79" Jun 11 16:17:53.987460 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:53.987432 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpfnk\" (UniqueName: \"kubernetes.io/projected/7be9c8f7-6bd2-4377-b06a-be5995d31d04-kube-api-access-bpfnk\") pod \"cert-manager-operator-controller-manager-57757f7cfc-rdr79\" (UID: \"7be9c8f7-6bd2-4377-b06a-be5995d31d04\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-rdr79" Jun 11 16:17:54.074495 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:54.074418 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-rdr79" Jun 11 16:17:54.203778 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:54.203750 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-rdr79"] Jun 11 16:17:54.207302 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:17:54.207270 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7be9c8f7_6bd2_4377_b06a_be5995d31d04.slice/crio-721d10826c37542c93ff148b94d5a58972ba45d007e37607c227a76a4258ea39 WatchSource:0}: Error finding container 721d10826c37542c93ff148b94d5a58972ba45d007e37607c227a76a4258ea39: Status 404 returned error can't find the container with id 721d10826c37542c93ff148b94d5a58972ba45d007e37607c227a76a4258ea39 Jun 11 16:17:54.487343 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:54.487312 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-rdr79" event={"ID":"7be9c8f7-6bd2-4377-b06a-be5995d31d04","Type":"ContainerStarted","Data":"721d10826c37542c93ff148b94d5a58972ba45d007e37607c227a76a4258ea39"} Jun 11 16:17:57.502473 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:57.502437 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-rdr79" event={"ID":"7be9c8f7-6bd2-4377-b06a-be5995d31d04","Type":"ContainerStarted","Data":"253307c4762afc526b65150c856577997b3b057ac7eb4d4db0b9164baf778dec"} Jun 11 16:17:57.524890 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:57.524834 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-rdr79" podStartSLOduration=2.257649334 podStartE2EDuration="4.524818792s" totalImagesPullingTime="2.267169458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:17:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:17:57.523732954 +0000 UTC m=+553.357845492" watchObservedRunningTime="2026-06-11 16:17:57.524818792 +0000 UTC m=+553.358931330" Jun 11 16:17:58.264792 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:58.264756 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn"] Jun 11 16:17:58.268146 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:58.268122 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" Jun 11 16:17:58.270606 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:58.270584 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qh6np\"" Jun 11 16:17:58.271366 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:58.271353 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 11 16:17:58.272565 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:58.272548 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 11 16:17:58.286526 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:58.286500 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn"] Jun 11 16:17:58.409146 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:58.409111 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f956\" (UniqueName: \"kubernetes.io/projected/09520f11-2409-4101-97f2-96ba37b39467-kube-api-access-4f956\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn\" (UID: \"09520f11-2409-4101-97f2-96ba37b39467\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" Jun 11 16:17:58.409336 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:58.409152 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09520f11-2409-4101-97f2-96ba37b39467-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn\" (UID: \"09520f11-2409-4101-97f2-96ba37b39467\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" Jun 11 16:17:58.409336 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:58.409179 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09520f11-2409-4101-97f2-96ba37b39467-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn\" (UID: \"09520f11-2409-4101-97f2-96ba37b39467\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" Jun 11 16:17:58.510555 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:58.510522 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f956\" (UniqueName: \"kubernetes.io/projected/09520f11-2409-4101-97f2-96ba37b39467-kube-api-access-4f956\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn\" (UID: \"09520f11-2409-4101-97f2-96ba37b39467\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" Jun 11 16:17:58.510555 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:58.510556 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09520f11-2409-4101-97f2-96ba37b39467-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn\" (UID: \"09520f11-2409-4101-97f2-96ba37b39467\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" Jun 11 16:17:58.510993 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:58.510587 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09520f11-2409-4101-97f2-96ba37b39467-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn\" (UID: \"09520f11-2409-4101-97f2-96ba37b39467\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" Jun 11 16:17:58.510993 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:58.510942 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09520f11-2409-4101-97f2-96ba37b39467-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn\" (UID: \"09520f11-2409-4101-97f2-96ba37b39467\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" Jun 11 16:17:58.510993 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:58.510948 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09520f11-2409-4101-97f2-96ba37b39467-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn\" (UID: \"09520f11-2409-4101-97f2-96ba37b39467\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" Jun 11 16:17:58.528382 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:58.528317 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f956\" (UniqueName: \"kubernetes.io/projected/09520f11-2409-4101-97f2-96ba37b39467-kube-api-access-4f956\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn\" (UID: \"09520f11-2409-4101-97f2-96ba37b39467\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" Jun 11 16:17:58.577309 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:58.577281 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" Jun 11 16:17:58.706782 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:58.706756 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn"] Jun 11 16:17:58.709212 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:17:58.709189 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09520f11_2409_4101_97f2_96ba37b39467.slice/crio-f1c048b291d6b5db97f640117cc47bb4fa9a3b40bce192a22f2cffec8e9f4f04 WatchSource:0}: Error finding container f1c048b291d6b5db97f640117cc47bb4fa9a3b40bce192a22f2cffec8e9f4f04: Status 404 returned error can't find the container with id f1c048b291d6b5db97f640117cc47bb4fa9a3b40bce192a22f2cffec8e9f4f04 Jun 11 16:17:59.511498 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:59.511459 2573 generic.go:356] "Generic (PLEG): container finished" podID="09520f11-2409-4101-97f2-96ba37b39467" containerID="a02026449affd1aa06ac2bd03d9c54346c222033d895c796381b08f3f97252ab" exitCode=0 Jun 11 16:17:59.511901 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:59.511538 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" event={"ID":"09520f11-2409-4101-97f2-96ba37b39467","Type":"ContainerDied","Data":"a02026449affd1aa06ac2bd03d9c54346c222033d895c796381b08f3f97252ab"} Jun 11 16:17:59.511901 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:17:59.511587 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" event={"ID":"09520f11-2409-4101-97f2-96ba37b39467","Type":"ContainerStarted","Data":"f1c048b291d6b5db97f640117cc47bb4fa9a3b40bce192a22f2cffec8e9f4f04"} Jun 11 16:18:00.188268 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:00.188235 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-b2m2z"] Jun 11 16:18:00.191323 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:00.191302 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-b2m2z" Jun 11 16:18:00.193693 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:00.193670 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 11 16:18:00.193812 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:00.193710 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-7lnvw\"" Jun 11 16:18:00.194356 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:00.194339 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 11 16:18:00.202599 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:00.202576 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-b2m2z"] Jun 11 16:18:00.326453 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:00.326415 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65c9ba1f-0811-4392-97a2-4ed13081efa6-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-b2m2z\" (UID: \"65c9ba1f-0811-4392-97a2-4ed13081efa6\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-b2m2z" Jun 11 16:18:00.326625 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:00.326481 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmtzh\" (UniqueName: \"kubernetes.io/projected/65c9ba1f-0811-4392-97a2-4ed13081efa6-kube-api-access-nmtzh\") pod \"cert-manager-webhook-7dc76cbb94-b2m2z\" (UID: \"65c9ba1f-0811-4392-97a2-4ed13081efa6\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-b2m2z" Jun 11 16:18:00.427744 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:00.427704 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65c9ba1f-0811-4392-97a2-4ed13081efa6-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-b2m2z\" (UID: \"65c9ba1f-0811-4392-97a2-4ed13081efa6\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-b2m2z" Jun 11 16:18:00.427947 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:00.427774 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmtzh\" (UniqueName: \"kubernetes.io/projected/65c9ba1f-0811-4392-97a2-4ed13081efa6-kube-api-access-nmtzh\") pod \"cert-manager-webhook-7dc76cbb94-b2m2z\" (UID: \"65c9ba1f-0811-4392-97a2-4ed13081efa6\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-b2m2z" Jun 11 16:18:00.440466 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:00.440390 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65c9ba1f-0811-4392-97a2-4ed13081efa6-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-b2m2z\" (UID: \"65c9ba1f-0811-4392-97a2-4ed13081efa6\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-b2m2z" Jun 11 16:18:00.440588 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:00.440499 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmtzh\" (UniqueName: \"kubernetes.io/projected/65c9ba1f-0811-4392-97a2-4ed13081efa6-kube-api-access-nmtzh\") pod \"cert-manager-webhook-7dc76cbb94-b2m2z\" (UID: \"65c9ba1f-0811-4392-97a2-4ed13081efa6\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-b2m2z" Jun 11 16:18:00.500507 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:00.500468 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-b2m2z" Jun 11 16:18:00.659424 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:00.659389 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-b2m2z"] Jun 11 16:18:00.662483 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:18:00.662451 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65c9ba1f_0811_4392_97a2_4ed13081efa6.slice/crio-b5aae7c3ec755538a57d65c4a5b4dfd7128ef09598f5bafe16044cc5baf0d1e4 WatchSource:0}: Error finding container b5aae7c3ec755538a57d65c4a5b4dfd7128ef09598f5bafe16044cc5baf0d1e4: Status 404 returned error can't find the container with id b5aae7c3ec755538a57d65c4a5b4dfd7128ef09598f5bafe16044cc5baf0d1e4 Jun 11 16:18:01.521788 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:01.521755 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-b2m2z" event={"ID":"65c9ba1f-0811-4392-97a2-4ed13081efa6","Type":"ContainerStarted","Data":"b5aae7c3ec755538a57d65c4a5b4dfd7128ef09598f5bafe16044cc5baf0d1e4"} Jun 11 16:18:02.528861 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:02.528808 2573 generic.go:356] "Generic (PLEG): container finished" podID="09520f11-2409-4101-97f2-96ba37b39467" containerID="25791a0ded601f9391bd0f1444260bc0b45962e59e597003748aa1fc55a5c505" exitCode=0 Jun 11 16:18:02.528861 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:02.528853 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" event={"ID":"09520f11-2409-4101-97f2-96ba37b39467","Type":"ContainerDied","Data":"25791a0ded601f9391bd0f1444260bc0b45962e59e597003748aa1fc55a5c505"} Jun 11 16:18:03.536874 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:03.536839 2573 generic.go:356] "Generic (PLEG): container finished" podID="09520f11-2409-4101-97f2-96ba37b39467" containerID="99ffc68aaa17ff7de853ab0dae3db48a97dc902dd45fe7fa176c607befef0754" exitCode=0 Jun 11 16:18:03.537329 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:03.536940 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" event={"ID":"09520f11-2409-4101-97f2-96ba37b39467","Type":"ContainerDied","Data":"99ffc68aaa17ff7de853ab0dae3db48a97dc902dd45fe7fa176c607befef0754"} Jun 11 16:18:04.542753 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:04.542719 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-b2m2z" event={"ID":"65c9ba1f-0811-4392-97a2-4ed13081efa6","Type":"ContainerStarted","Data":"7b71b124091a1ee7ab78681ae526fa0015d696468c506afd5c200a30bb204815"} Jun 11 16:18:04.543254 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:04.542945 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-b2m2z" Jun 11 16:18:04.561831 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:04.561781 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-b2m2z" podStartSLOduration=1.590477836 podStartE2EDuration="4.56176295s" totalImagesPullingTime="2.971285114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:18:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:18:04.559297859 +0000 UTC m=+560.393410398" watchObservedRunningTime="2026-06-11 16:18:04.56176295 +0000 UTC m=+560.395875488" Jun 11 16:18:04.668839 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:04.668814 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" Jun 11 16:18:04.766968 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:04.766907 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09520f11-2409-4101-97f2-96ba37b39467-util\") pod \"09520f11-2409-4101-97f2-96ba37b39467\" (UID: \"09520f11-2409-4101-97f2-96ba37b39467\") " Jun 11 16:18:04.767133 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:04.766976 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09520f11-2409-4101-97f2-96ba37b39467-bundle\") pod \"09520f11-2409-4101-97f2-96ba37b39467\" (UID: \"09520f11-2409-4101-97f2-96ba37b39467\") " Jun 11 16:18:04.767133 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:04.767011 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f956\" (UniqueName: \"kubernetes.io/projected/09520f11-2409-4101-97f2-96ba37b39467-kube-api-access-4f956\") pod \"09520f11-2409-4101-97f2-96ba37b39467\" (UID: \"09520f11-2409-4101-97f2-96ba37b39467\") " Jun 11 16:18:04.767416 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:04.767388 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09520f11-2409-4101-97f2-96ba37b39467-bundle" (OuterVolumeSpecName: "bundle") pod "09520f11-2409-4101-97f2-96ba37b39467" (UID: "09520f11-2409-4101-97f2-96ba37b39467"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:18:04.769232 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:04.769207 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09520f11-2409-4101-97f2-96ba37b39467-kube-api-access-4f956" (OuterVolumeSpecName: "kube-api-access-4f956") pod "09520f11-2409-4101-97f2-96ba37b39467" (UID: "09520f11-2409-4101-97f2-96ba37b39467"). InnerVolumeSpecName "kube-api-access-4f956". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:18:04.773230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:04.773204 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09520f11-2409-4101-97f2-96ba37b39467-util" (OuterVolumeSpecName: "util") pod "09520f11-2409-4101-97f2-96ba37b39467" (UID: "09520f11-2409-4101-97f2-96ba37b39467"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:18:04.868230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:04.868142 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f956\" (UniqueName: \"kubernetes.io/projected/09520f11-2409-4101-97f2-96ba37b39467-kube-api-access-4f956\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:18:04.868230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:04.868173 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/09520f11-2409-4101-97f2-96ba37b39467-util\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:18:04.868230 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:04.868185 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/09520f11-2409-4101-97f2-96ba37b39467-bundle\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:18:05.548677 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:05.548643 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" Jun 11 16:18:05.548677 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:05.548661 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fjzbxn" event={"ID":"09520f11-2409-4101-97f2-96ba37b39467","Type":"ContainerDied","Data":"f1c048b291d6b5db97f640117cc47bb4fa9a3b40bce192a22f2cffec8e9f4f04"} Jun 11 16:18:05.549223 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:05.548732 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1c048b291d6b5db97f640117cc47bb4fa9a3b40bce192a22f2cffec8e9f4f04" Jun 11 16:18:10.064279 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.064241 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-59c77684cc-jsz74"] Jun 11 16:18:10.064672 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.064538 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09520f11-2409-4101-97f2-96ba37b39467" containerName="pull" Jun 11 16:18:10.064672 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.064550 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="09520f11-2409-4101-97f2-96ba37b39467" containerName="pull" Jun 11 16:18:10.064672 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.064563 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09520f11-2409-4101-97f2-96ba37b39467" containerName="util" Jun 11 16:18:10.064672 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.064569 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="09520f11-2409-4101-97f2-96ba37b39467" containerName="util" Jun 11 16:18:10.064672 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.064575 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09520f11-2409-4101-97f2-96ba37b39467" containerName="extract" Jun 11 16:18:10.064672 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.064580 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="09520f11-2409-4101-97f2-96ba37b39467" containerName="extract" Jun 11 16:18:10.064672 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.064630 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="09520f11-2409-4101-97f2-96ba37b39467" containerName="extract" Jun 11 16:18:10.068453 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.068434 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-jsz74" Jun 11 16:18:10.070582 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.070555 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-f8l9k\"" Jun 11 16:18:10.077105 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.077083 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-jsz74"] Jun 11 16:18:10.207644 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.207596 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8476f662-1128-4fa0-af89-c340c0ca6381-bound-sa-token\") pod \"cert-manager-59c77684cc-jsz74\" (UID: \"8476f662-1128-4fa0-af89-c340c0ca6381\") " pod="cert-manager/cert-manager-59c77684cc-jsz74" Jun 11 16:18:10.207824 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.207665 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ds8t\" (UniqueName: \"kubernetes.io/projected/8476f662-1128-4fa0-af89-c340c0ca6381-kube-api-access-6ds8t\") pod \"cert-manager-59c77684cc-jsz74\" (UID: \"8476f662-1128-4fa0-af89-c340c0ca6381\") " pod="cert-manager/cert-manager-59c77684cc-jsz74" Jun 11 16:18:10.308319 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.308281 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8476f662-1128-4fa0-af89-c340c0ca6381-bound-sa-token\") pod \"cert-manager-59c77684cc-jsz74\" (UID: \"8476f662-1128-4fa0-af89-c340c0ca6381\") " pod="cert-manager/cert-manager-59c77684cc-jsz74" Jun 11 16:18:10.308501 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.308350 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ds8t\" (UniqueName: \"kubernetes.io/projected/8476f662-1128-4fa0-af89-c340c0ca6381-kube-api-access-6ds8t\") pod \"cert-manager-59c77684cc-jsz74\" (UID: \"8476f662-1128-4fa0-af89-c340c0ca6381\") " pod="cert-manager/cert-manager-59c77684cc-jsz74" Jun 11 16:18:10.316682 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.316618 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8476f662-1128-4fa0-af89-c340c0ca6381-bound-sa-token\") pod \"cert-manager-59c77684cc-jsz74\" (UID: \"8476f662-1128-4fa0-af89-c340c0ca6381\") " pod="cert-manager/cert-manager-59c77684cc-jsz74" Jun 11 16:18:10.316785 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.316770 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ds8t\" (UniqueName: \"kubernetes.io/projected/8476f662-1128-4fa0-af89-c340c0ca6381-kube-api-access-6ds8t\") pod \"cert-manager-59c77684cc-jsz74\" (UID: \"8476f662-1128-4fa0-af89-c340c0ca6381\") " pod="cert-manager/cert-manager-59c77684cc-jsz74" Jun 11 16:18:10.377683 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.377648 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-jsz74" Jun 11 16:18:10.510817 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.510784 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-jsz74"] Jun 11 16:18:10.513224 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:18:10.513195 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8476f662_1128_4fa0_af89_c340c0ca6381.slice/crio-a438b67f36496ceecc70e8b22e077d1ed922ebd8d6bcce1e8117c1fb547c7076 WatchSource:0}: Error finding container a438b67f36496ceecc70e8b22e077d1ed922ebd8d6bcce1e8117c1fb547c7076: Status 404 returned error can't find the container with id a438b67f36496ceecc70e8b22e077d1ed922ebd8d6bcce1e8117c1fb547c7076 Jun 11 16:18:10.551062 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.551040 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-b2m2z" Jun 11 16:18:10.574898 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:10.574867 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-jsz74" event={"ID":"8476f662-1128-4fa0-af89-c340c0ca6381","Type":"ContainerStarted","Data":"a438b67f36496ceecc70e8b22e077d1ed922ebd8d6bcce1e8117c1fb547c7076"} Jun 11 16:18:11.582280 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:11.582244 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-jsz74" event={"ID":"8476f662-1128-4fa0-af89-c340c0ca6381","Type":"ContainerStarted","Data":"40d47bf5fbf58bdfe1966f17a2ec616ffc12a191042e59d63dd4db367d60eb07"} Jun 11 16:18:11.599954 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:11.599888 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-59c77684cc-jsz74" podStartSLOduration=1.599871498 podStartE2EDuration="1.599871498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:18:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:18:11.598129385 +0000 UTC m=+567.432241923" watchObservedRunningTime="2026-06-11 16:18:11.599871498 +0000 UTC m=+567.433984036" Jun 11 16:18:21.922391 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:21.922349 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm"] Jun 11 16:18:21.927014 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:21.926994 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" Jun 11 16:18:21.929366 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:21.929341 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 11 16:18:21.929366 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:21.929355 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qh6np\"" Jun 11 16:18:21.929963 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:21.929945 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 11 16:18:21.934029 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:21.934003 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm"] Jun 11 16:18:21.993297 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:21.993244 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm\" (UID: \"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" Jun 11 16:18:21.993297 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:21.993311 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm\" (UID: \"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" Jun 11 16:18:21.993525 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:21.993346 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6cz7\" (UniqueName: \"kubernetes.io/projected/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-kube-api-access-v6cz7\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm\" (UID: \"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" Jun 11 16:18:22.094628 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:22.094584 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm\" (UID: \"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" Jun 11 16:18:22.094767 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:22.094640 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm\" (UID: \"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" Jun 11 16:18:22.094767 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:22.094752 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6cz7\" (UniqueName: \"kubernetes.io/projected/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-kube-api-access-v6cz7\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm\" (UID: \"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" Jun 11 16:18:22.095056 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:22.095031 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm\" (UID: \"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" Jun 11 16:18:22.095187 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:22.095093 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm\" (UID: \"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" Jun 11 16:18:22.103229 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:22.103207 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6cz7\" (UniqueName: \"kubernetes.io/projected/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-kube-api-access-v6cz7\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm\" (UID: \"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" Jun 11 16:18:22.237601 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:22.237511 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" Jun 11 16:18:22.368740 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:22.368714 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm"] Jun 11 16:18:22.371452 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:18:22.371430 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e3c0a88_eae7_406b_a9b9_fc8dbfd0fc53.slice/crio-1ef92954e1f8de39ba17df331f438d826cc2d27c3bf76a22c9e3a795dcfd0537 WatchSource:0}: Error finding container 1ef92954e1f8de39ba17df331f438d826cc2d27c3bf76a22c9e3a795dcfd0537: Status 404 returned error can't find the container with id 1ef92954e1f8de39ba17df331f438d826cc2d27c3bf76a22c9e3a795dcfd0537 Jun 11 16:18:22.628951 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:22.628844 2573 generic.go:356] "Generic (PLEG): container finished" podID="3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53" containerID="1b9d61a16e60ca2188c24fd2d192bdad3130d53ccfa8560819f96a7d8ba74287" exitCode=0 Jun 11 16:18:22.629110 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:22.628951 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" event={"ID":"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53","Type":"ContainerDied","Data":"1b9d61a16e60ca2188c24fd2d192bdad3130d53ccfa8560819f96a7d8ba74287"} Jun 11 16:18:22.629110 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:22.629010 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" event={"ID":"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53","Type":"ContainerStarted","Data":"1ef92954e1f8de39ba17df331f438d826cc2d27c3bf76a22c9e3a795dcfd0537"} Jun 11 16:18:23.634799 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:23.634766 2573 generic.go:356] "Generic (PLEG): container finished" podID="3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53" containerID="7f652325852b48be54fbb317486f335ee36966f336a95e6c197519e938e85fd1" exitCode=0 Jun 11 16:18:23.635186 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:23.634810 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" event={"ID":"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53","Type":"ContainerDied","Data":"7f652325852b48be54fbb317486f335ee36966f336a95e6c197519e938e85fd1"} Jun 11 16:18:24.640827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:24.640793 2573 generic.go:356] "Generic (PLEG): container finished" podID="3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53" containerID="8ea9b170d57f15f2fe66f79286d20191b10247dce083db1b90499d700ba2f1c9" exitCode=0 Jun 11 16:18:24.641245 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:24.640854 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" event={"ID":"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53","Type":"ContainerDied","Data":"8ea9b170d57f15f2fe66f79286d20191b10247dce083db1b90499d700ba2f1c9"} Jun 11 16:18:25.769475 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:25.769452 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" Jun 11 16:18:25.824633 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:25.824603 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-util\") pod \"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53\" (UID: \"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53\") " Jun 11 16:18:25.824814 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:25.824643 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6cz7\" (UniqueName: \"kubernetes.io/projected/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-kube-api-access-v6cz7\") pod \"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53\" (UID: \"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53\") " Jun 11 16:18:25.824814 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:25.824689 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-bundle\") pod \"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53\" (UID: \"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53\") " Jun 11 16:18:25.825458 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:25.825366 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-bundle" (OuterVolumeSpecName: "bundle") pod "3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53" (UID: "3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:18:25.826832 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:25.826810 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-kube-api-access-v6cz7" (OuterVolumeSpecName: "kube-api-access-v6cz7") pod "3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53" (UID: "3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53"). InnerVolumeSpecName "kube-api-access-v6cz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:18:25.829819 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:25.829781 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-util" (OuterVolumeSpecName: "util") pod "3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53" (UID: "3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:18:25.926081 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:25.926044 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-util\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:18:25.926081 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:25.926075 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6cz7\" (UniqueName: \"kubernetes.io/projected/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-kube-api-access-v6cz7\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:18:25.926264 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:25.926091 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53-bundle\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:18:26.651117 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:26.651071 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" event={"ID":"3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53","Type":"ContainerDied","Data":"1ef92954e1f8de39ba17df331f438d826cc2d27c3bf76a22c9e3a795dcfd0537"} Jun 11 16:18:26.651276 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:26.651132 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1ef92954e1f8de39ba17df331f438d826cc2d27c3bf76a22c9e3a795dcfd0537" Jun 11 16:18:26.651276 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:26.651086 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdg4rqm" Jun 11 16:18:36.042478 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.042438 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd"] Jun 11 16:18:36.042860 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.042740 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53" containerName="util" Jun 11 16:18:36.042860 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.042750 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53" containerName="util" Jun 11 16:18:36.042860 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.042760 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53" containerName="extract" Jun 11 16:18:36.042860 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.042765 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53" containerName="extract" Jun 11 16:18:36.042860 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.042775 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53" containerName="pull" Jun 11 16:18:36.042860 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.042780 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53" containerName="pull" Jun 11 16:18:36.042860 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.042836 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e3c0a88-eae7-406b-a9b9-fc8dbfd0fc53" containerName="extract" Jun 11 16:18:36.045793 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.045777 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:36.055599 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.055578 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 11 16:18:36.055745 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.055616 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 11 16:18:36.055745 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.055615 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 11 16:18:36.055745 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.055684 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-clg2l\"" Jun 11 16:18:36.055745 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.055637 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 11 16:18:36.077450 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.077424 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd"] Jun 11 16:18:36.103153 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.103121 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppjwc\" (UniqueName: \"kubernetes.io/projected/640fbc8e-b66f-4a95-b330-e965be57d616-kube-api-access-ppjwc\") pod \"opendatahub-operator-controller-manager-fb875574b-4skmd\" (UID: \"640fbc8e-b66f-4a95-b330-e965be57d616\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:36.103296 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.103171 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/640fbc8e-b66f-4a95-b330-e965be57d616-webhook-cert\") pod \"opendatahub-operator-controller-manager-fb875574b-4skmd\" (UID: \"640fbc8e-b66f-4a95-b330-e965be57d616\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:36.103296 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.103198 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/640fbc8e-b66f-4a95-b330-e965be57d616-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fb875574b-4skmd\" (UID: \"640fbc8e-b66f-4a95-b330-e965be57d616\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:36.103296 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.103216 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/640fbc8e-b66f-4a95-b330-e965be57d616-tmp\") pod \"opendatahub-operator-controller-manager-fb875574b-4skmd\" (UID: \"640fbc8e-b66f-4a95-b330-e965be57d616\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:36.103296 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.103258 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/640fbc8e-b66f-4a95-b330-e965be57d616-manifests\") pod \"opendatahub-operator-controller-manager-fb875574b-4skmd\" (UID: \"640fbc8e-b66f-4a95-b330-e965be57d616\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:36.203801 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.203770 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/640fbc8e-b66f-4a95-b330-e965be57d616-manifests\") pod \"opendatahub-operator-controller-manager-fb875574b-4skmd\" (UID: \"640fbc8e-b66f-4a95-b330-e965be57d616\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:36.203801 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.203804 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppjwc\" (UniqueName: \"kubernetes.io/projected/640fbc8e-b66f-4a95-b330-e965be57d616-kube-api-access-ppjwc\") pod \"opendatahub-operator-controller-manager-fb875574b-4skmd\" (UID: \"640fbc8e-b66f-4a95-b330-e965be57d616\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:36.204054 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.203836 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/640fbc8e-b66f-4a95-b330-e965be57d616-webhook-cert\") pod \"opendatahub-operator-controller-manager-fb875574b-4skmd\" (UID: \"640fbc8e-b66f-4a95-b330-e965be57d616\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:36.204054 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.203887 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/640fbc8e-b66f-4a95-b330-e965be57d616-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fb875574b-4skmd\" (UID: \"640fbc8e-b66f-4a95-b330-e965be57d616\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:36.204054 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.203942 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/640fbc8e-b66f-4a95-b330-e965be57d616-tmp\") pod \"opendatahub-operator-controller-manager-fb875574b-4skmd\" (UID: \"640fbc8e-b66f-4a95-b330-e965be57d616\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:36.204208 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.204190 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/640fbc8e-b66f-4a95-b330-e965be57d616-manifests\") pod \"opendatahub-operator-controller-manager-fb875574b-4skmd\" (UID: \"640fbc8e-b66f-4a95-b330-e965be57d616\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:36.206316 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.206287 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/640fbc8e-b66f-4a95-b330-e965be57d616-tmp\") pod \"opendatahub-operator-controller-manager-fb875574b-4skmd\" (UID: \"640fbc8e-b66f-4a95-b330-e965be57d616\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:36.206522 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.206506 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/640fbc8e-b66f-4a95-b330-e965be57d616-webhook-cert\") pod \"opendatahub-operator-controller-manager-fb875574b-4skmd\" (UID: \"640fbc8e-b66f-4a95-b330-e965be57d616\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:36.206563 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.206533 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/640fbc8e-b66f-4a95-b330-e965be57d616-apiservice-cert\") pod \"opendatahub-operator-controller-manager-fb875574b-4skmd\" (UID: \"640fbc8e-b66f-4a95-b330-e965be57d616\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:36.218843 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.218822 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppjwc\" (UniqueName: \"kubernetes.io/projected/640fbc8e-b66f-4a95-b330-e965be57d616-kube-api-access-ppjwc\") pod \"opendatahub-operator-controller-manager-fb875574b-4skmd\" (UID: \"640fbc8e-b66f-4a95-b330-e965be57d616\") " pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:36.355732 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.355646 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:36.497232 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.497191 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd"] Jun 11 16:18:36.500008 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:18:36.499979 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod640fbc8e_b66f_4a95_b330_e965be57d616.slice/crio-7718e70eac089e120700f60fe622ef6565c526996739bd82e8307d276007d876 WatchSource:0}: Error finding container 7718e70eac089e120700f60fe622ef6565c526996739bd82e8307d276007d876: Status 404 returned error can't find the container with id 7718e70eac089e120700f60fe622ef6565c526996739bd82e8307d276007d876 Jun 11 16:18:36.693899 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:36.693863 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" event={"ID":"640fbc8e-b66f-4a95-b330-e965be57d616","Type":"ContainerStarted","Data":"7718e70eac089e120700f60fe622ef6565c526996739bd82e8307d276007d876"} Jun 11 16:18:39.710423 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:39.710395 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" event={"ID":"640fbc8e-b66f-4a95-b330-e965be57d616","Type":"ContainerStarted","Data":"0573e9ee4bcf3664bd81e2965d2a105b5e0cbe2a3d8b90f5383ca656794df200"} Jun 11 16:18:40.716559 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:40.716517 2573 generic.go:356] "Generic (PLEG): container finished" podID="640fbc8e-b66f-4a95-b330-e965be57d616" containerID="0573e9ee4bcf3664bd81e2965d2a105b5e0cbe2a3d8b90f5383ca656794df200" exitCode=0 Jun 11 16:18:40.716878 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:40.716568 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" event={"ID":"640fbc8e-b66f-4a95-b330-e965be57d616","Type":"ContainerDied","Data":"0573e9ee4bcf3664bd81e2965d2a105b5e0cbe2a3d8b90f5383ca656794df200"} Jun 11 16:18:41.722795 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:41.722755 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" event={"ID":"640fbc8e-b66f-4a95-b330-e965be57d616","Type":"ContainerStarted","Data":"67d4efb1ad1424f521e9a29b057a207b0604c2a8ec3996cde96d38756356c4d1"} Jun 11 16:18:41.723256 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:41.722892 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:41.748720 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:41.748667 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" podStartSLOduration=1.151984679 podStartE2EDuration="5.748650031s" totalImagesPullingTime="3.634034987s" totalInitContainerRuntime="962.630365ms" isStatefulPod=false podCreationTimestamp="2026-06-11 16:18:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:18:41.747006836 +0000 UTC m=+597.581119375" watchObservedRunningTime="2026-06-11 16:18:41.748650031 +0000 UTC m=+597.582762639" Jun 11 16:18:44.545324 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:44.545292 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p7nvz_7acea643-2978-4893-a6a9-bf88a11b531b/ovn-acl-logging/0.log" Jun 11 16:18:44.548025 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:44.548005 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p7nvz_7acea643-2978-4893-a6a9-bf88a11b531b/ovn-acl-logging/0.log" Jun 11 16:18:52.729896 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:52.729863 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-fb875574b-4skmd" Jun 11 16:18:55.911190 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:55.911151 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6bbbf84858-z99qw"] Jun 11 16:18:55.914260 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:55.914243 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bbbf84858-z99qw" Jun 11 16:18:55.917103 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:55.917083 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 11 16:18:55.920828 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:55.920600 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 11 16:18:55.951903 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:55.951871 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bbbf84858-z99qw"] Jun 11 16:18:56.066711 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.066671 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9s4n\" (UniqueName: \"kubernetes.io/projected/fa0ac318-b430-4185-a0e4-79f600cf3169-kube-api-access-m9s4n\") pod \"kube-auth-proxy-6bbbf84858-z99qw\" (UID: \"fa0ac318-b430-4185-a0e4-79f600cf3169\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-z99qw" Jun 11 16:18:56.066884 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.066721 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa0ac318-b430-4185-a0e4-79f600cf3169-tls-certs\") pod \"kube-auth-proxy-6bbbf84858-z99qw\" (UID: \"fa0ac318-b430-4185-a0e4-79f600cf3169\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-z99qw" Jun 11 16:18:56.066884 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.066748 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa0ac318-b430-4185-a0e4-79f600cf3169-tmp\") pod \"kube-auth-proxy-6bbbf84858-z99qw\" (UID: \"fa0ac318-b430-4185-a0e4-79f600cf3169\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-z99qw" Jun 11 16:18:56.167872 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.167781 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9s4n\" (UniqueName: \"kubernetes.io/projected/fa0ac318-b430-4185-a0e4-79f600cf3169-kube-api-access-m9s4n\") pod \"kube-auth-proxy-6bbbf84858-z99qw\" (UID: \"fa0ac318-b430-4185-a0e4-79f600cf3169\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-z99qw" Jun 11 16:18:56.167872 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.167836 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa0ac318-b430-4185-a0e4-79f600cf3169-tls-certs\") pod \"kube-auth-proxy-6bbbf84858-z99qw\" (UID: \"fa0ac318-b430-4185-a0e4-79f600cf3169\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-z99qw" Jun 11 16:18:56.167872 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.167866 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa0ac318-b430-4185-a0e4-79f600cf3169-tmp\") pod \"kube-auth-proxy-6bbbf84858-z99qw\" (UID: \"fa0ac318-b430-4185-a0e4-79f600cf3169\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-z99qw" Jun 11 16:18:56.170392 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.170367 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa0ac318-b430-4185-a0e4-79f600cf3169-tmp\") pod \"kube-auth-proxy-6bbbf84858-z99qw\" (UID: \"fa0ac318-b430-4185-a0e4-79f600cf3169\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-z99qw" Jun 11 16:18:56.170519 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.170508 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa0ac318-b430-4185-a0e4-79f600cf3169-tls-certs\") pod \"kube-auth-proxy-6bbbf84858-z99qw\" (UID: \"fa0ac318-b430-4185-a0e4-79f600cf3169\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-z99qw" Jun 11 16:18:56.201957 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.201906 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9s4n\" (UniqueName: \"kubernetes.io/projected/fa0ac318-b430-4185-a0e4-79f600cf3169-kube-api-access-m9s4n\") pod \"kube-auth-proxy-6bbbf84858-z99qw\" (UID: \"fa0ac318-b430-4185-a0e4-79f600cf3169\") " pod="openshift-ingress/kube-auth-proxy-6bbbf84858-z99qw" Jun 11 16:18:56.224787 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.224756 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6bbbf84858-z99qw" Jun 11 16:18:56.276706 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.276665 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-rpwx9"] Jun 11 16:18:56.280419 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.280396 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-rpwx9" Jun 11 16:18:56.286914 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.286891 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-4f6pm\"" Jun 11 16:18:56.289103 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.289080 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 11 16:18:56.297609 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.297577 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-rpwx9"] Jun 11 16:18:56.370262 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.370235 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2787e2b3-3de4-400b-962f-2685f017b12e-cert\") pod \"odh-model-controller-6dc94f689f-rpwx9\" (UID: \"2787e2b3-3de4-400b-962f-2685f017b12e\") " pod="opendatahub/odh-model-controller-6dc94f689f-rpwx9" Jun 11 16:18:56.370262 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.370265 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mght9\" (UniqueName: \"kubernetes.io/projected/2787e2b3-3de4-400b-962f-2685f017b12e-kube-api-access-mght9\") pod \"odh-model-controller-6dc94f689f-rpwx9\" (UID: \"2787e2b3-3de4-400b-962f-2685f017b12e\") " pod="opendatahub/odh-model-controller-6dc94f689f-rpwx9" Jun 11 16:18:56.381108 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.381086 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6bbbf84858-z99qw"] Jun 11 16:18:56.383046 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:18:56.383022 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa0ac318_b430_4185_a0e4_79f600cf3169.slice/crio-ced705b5fead935ba355da6598c647680da6e7761bb571cd94c5fefce7bee569 WatchSource:0}: Error finding container ced705b5fead935ba355da6598c647680da6e7761bb571cd94c5fefce7bee569: Status 404 returned error can't find the container with id ced705b5fead935ba355da6598c647680da6e7761bb571cd94c5fefce7bee569 Jun 11 16:18:56.384657 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.384642 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 16:18:56.471045 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.470968 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2787e2b3-3de4-400b-962f-2685f017b12e-cert\") pod \"odh-model-controller-6dc94f689f-rpwx9\" (UID: \"2787e2b3-3de4-400b-962f-2685f017b12e\") " pod="opendatahub/odh-model-controller-6dc94f689f-rpwx9" Jun 11 16:18:56.471045 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.470999 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mght9\" (UniqueName: \"kubernetes.io/projected/2787e2b3-3de4-400b-962f-2685f017b12e-kube-api-access-mght9\") pod \"odh-model-controller-6dc94f689f-rpwx9\" (UID: \"2787e2b3-3de4-400b-962f-2685f017b12e\") " pod="opendatahub/odh-model-controller-6dc94f689f-rpwx9" Jun 11 16:18:56.473482 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.473453 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2787e2b3-3de4-400b-962f-2685f017b12e-cert\") pod \"odh-model-controller-6dc94f689f-rpwx9\" (UID: \"2787e2b3-3de4-400b-962f-2685f017b12e\") " pod="opendatahub/odh-model-controller-6dc94f689f-rpwx9" Jun 11 16:18:56.490126 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.490104 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mght9\" (UniqueName: \"kubernetes.io/projected/2787e2b3-3de4-400b-962f-2685f017b12e-kube-api-access-mght9\") pod \"odh-model-controller-6dc94f689f-rpwx9\" (UID: \"2787e2b3-3de4-400b-962f-2685f017b12e\") " pod="opendatahub/odh-model-controller-6dc94f689f-rpwx9" Jun 11 16:18:56.591823 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.591790 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-rpwx9" Jun 11 16:18:56.728544 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.728358 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-rpwx9"] Jun 11 16:18:56.790110 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.790073 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bbbf84858-z99qw" event={"ID":"fa0ac318-b430-4185-a0e4-79f600cf3169","Type":"ContainerStarted","Data":"ced705b5fead935ba355da6598c647680da6e7761bb571cd94c5fefce7bee569"} Jun 11 16:18:56.791139 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:56.791113 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-rpwx9" event={"ID":"2787e2b3-3de4-400b-962f-2685f017b12e","Type":"ContainerStarted","Data":"96006a747c468596ea91e9bda1d474b7363f47367f57894216ff21bbd22b2b79"} Jun 11 16:18:57.111258 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.111168 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-xmkq6"] Jun 11 16:18:57.114776 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.114751 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-xmkq6" Jun 11 16:18:57.119227 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.119200 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 11 16:18:57.119367 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.119261 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-7rgpl\"" Jun 11 16:18:57.141532 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.141429 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-xmkq6"] Jun 11 16:18:57.178508 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.178465 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ce98183-5dc0-4034-ab51-718722ff4da0-tls-certs\") pod \"model-serving-api-5dd488fc48-xmkq6\" (UID: \"8ce98183-5dc0-4034-ab51-718722ff4da0\") " pod="opendatahub/model-serving-api-5dd488fc48-xmkq6" Jun 11 16:18:57.178681 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.178570 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx8w2\" (UniqueName: \"kubernetes.io/projected/8ce98183-5dc0-4034-ab51-718722ff4da0-kube-api-access-qx8w2\") pod \"model-serving-api-5dd488fc48-xmkq6\" (UID: \"8ce98183-5dc0-4034-ab51-718722ff4da0\") " pod="opendatahub/model-serving-api-5dd488fc48-xmkq6" Jun 11 16:18:57.279848 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.279807 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ce98183-5dc0-4034-ab51-718722ff4da0-tls-certs\") pod \"model-serving-api-5dd488fc48-xmkq6\" (UID: \"8ce98183-5dc0-4034-ab51-718722ff4da0\") " pod="opendatahub/model-serving-api-5dd488fc48-xmkq6" Jun 11 16:18:57.280058 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.279963 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx8w2\" (UniqueName: \"kubernetes.io/projected/8ce98183-5dc0-4034-ab51-718722ff4da0-kube-api-access-qx8w2\") pod \"model-serving-api-5dd488fc48-xmkq6\" (UID: \"8ce98183-5dc0-4034-ab51-718722ff4da0\") " pod="opendatahub/model-serving-api-5dd488fc48-xmkq6" Jun 11 16:18:57.280058 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:18:57.279992 2573 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jun 11 16:18:57.280171 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:18:57.280081 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ce98183-5dc0-4034-ab51-718722ff4da0-tls-certs podName:8ce98183-5dc0-4034-ab51-718722ff4da0 nodeName:}" failed. No retries permitted until 2026-06-11 16:18:57.780058589 +0000 UTC m=+613.614171110 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8ce98183-5dc0-4034-ab51-718722ff4da0-tls-certs") pod "model-serving-api-5dd488fc48-xmkq6" (UID: "8ce98183-5dc0-4034-ab51-718722ff4da0") : secret "model-serving-api-tls" not found Jun 11 16:18:57.291337 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.291309 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx8w2\" (UniqueName: \"kubernetes.io/projected/8ce98183-5dc0-4034-ab51-718722ff4da0-kube-api-access-qx8w2\") pod \"model-serving-api-5dd488fc48-xmkq6\" (UID: \"8ce98183-5dc0-4034-ab51-718722ff4da0\") " pod="opendatahub/model-serving-api-5dd488fc48-xmkq6" Jun 11 16:18:57.617048 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.617011 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs"] Jun 11 16:18:57.621144 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.621124 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" Jun 11 16:18:57.623342 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.623296 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 11 16:18:57.623574 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.623556 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qh6np\"" Jun 11 16:18:57.623641 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.623596 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 11 16:18:57.629159 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.629136 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs"] Jun 11 16:18:57.683460 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.683352 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51d361a5-3bbe-4910-8e53-79fc4dc35153-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs\" (UID: \"51d361a5-3bbe-4910-8e53-79fc4dc35153\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" Jun 11 16:18:57.683460 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.683398 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq744\" (UniqueName: \"kubernetes.io/projected/51d361a5-3bbe-4910-8e53-79fc4dc35153-kube-api-access-jq744\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs\" (UID: \"51d361a5-3bbe-4910-8e53-79fc4dc35153\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" Jun 11 16:18:57.683743 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.683531 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51d361a5-3bbe-4910-8e53-79fc4dc35153-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs\" (UID: \"51d361a5-3bbe-4910-8e53-79fc4dc35153\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" Jun 11 16:18:57.784276 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.784236 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51d361a5-3bbe-4910-8e53-79fc4dc35153-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs\" (UID: \"51d361a5-3bbe-4910-8e53-79fc4dc35153\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" Jun 11 16:18:57.784495 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.784331 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51d361a5-3bbe-4910-8e53-79fc4dc35153-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs\" (UID: \"51d361a5-3bbe-4910-8e53-79fc4dc35153\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" Jun 11 16:18:57.784495 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.784361 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq744\" (UniqueName: \"kubernetes.io/projected/51d361a5-3bbe-4910-8e53-79fc4dc35153-kube-api-access-jq744\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs\" (UID: \"51d361a5-3bbe-4910-8e53-79fc4dc35153\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" Jun 11 16:18:57.784495 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.784397 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ce98183-5dc0-4034-ab51-718722ff4da0-tls-certs\") pod \"model-serving-api-5dd488fc48-xmkq6\" (UID: \"8ce98183-5dc0-4034-ab51-718722ff4da0\") " pod="opendatahub/model-serving-api-5dd488fc48-xmkq6" Jun 11 16:18:57.785378 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.785224 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51d361a5-3bbe-4910-8e53-79fc4dc35153-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs\" (UID: \"51d361a5-3bbe-4910-8e53-79fc4dc35153\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" Jun 11 16:18:57.785488 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.785463 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51d361a5-3bbe-4910-8e53-79fc4dc35153-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs\" (UID: \"51d361a5-3bbe-4910-8e53-79fc4dc35153\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" Jun 11 16:18:57.788819 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.788775 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ce98183-5dc0-4034-ab51-718722ff4da0-tls-certs\") pod \"model-serving-api-5dd488fc48-xmkq6\" (UID: \"8ce98183-5dc0-4034-ab51-718722ff4da0\") " pod="opendatahub/model-serving-api-5dd488fc48-xmkq6" Jun 11 16:18:57.797281 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.797232 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq744\" (UniqueName: \"kubernetes.io/projected/51d361a5-3bbe-4910-8e53-79fc4dc35153-kube-api-access-jq744\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs\" (UID: \"51d361a5-3bbe-4910-8e53-79fc4dc35153\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" Jun 11 16:18:57.936099 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:57.936043 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" Jun 11 16:18:58.035028 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:18:58.034762 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-xmkq6" Jun 11 16:19:00.797138 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:00.797094 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-xmkq6"] Jun 11 16:19:00.799413 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:19:00.799361 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ce98183_5dc0_4034_ab51_718722ff4da0.slice/crio-dd6af1dcad2cb5fa24b5b26db417dabb1fe6c4e0b395d70edcc07e073bd31fb7 WatchSource:0}: Error finding container dd6af1dcad2cb5fa24b5b26db417dabb1fe6c4e0b395d70edcc07e073bd31fb7: Status 404 returned error can't find the container with id dd6af1dcad2cb5fa24b5b26db417dabb1fe6c4e0b395d70edcc07e073bd31fb7 Jun 11 16:19:00.813757 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:00.813730 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6bbbf84858-z99qw" event={"ID":"fa0ac318-b430-4185-a0e4-79f600cf3169","Type":"ContainerStarted","Data":"6ffd21079aa2334d7ad390848b16b22203978a9005961c93888003488acf1a29"} Jun 11 16:19:00.814841 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:00.814818 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-xmkq6" event={"ID":"8ce98183-5dc0-4034-ab51-718722ff4da0","Type":"ContainerStarted","Data":"dd6af1dcad2cb5fa24b5b26db417dabb1fe6c4e0b395d70edcc07e073bd31fb7"} Jun 11 16:19:00.816247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:00.816228 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-rpwx9" event={"ID":"2787e2b3-3de4-400b-962f-2685f017b12e","Type":"ContainerStarted","Data":"093a28b29a5663af53f05bf7738119d410a18fb91e807b671ca3d172b76874b9"} Jun 11 16:19:00.816380 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:00.816364 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-6dc94f689f-rpwx9" Jun 11 16:19:00.822897 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:00.822875 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs"] Jun 11 16:19:00.825440 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:19:00.825415 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51d361a5_3bbe_4910_8e53_79fc4dc35153.slice/crio-9ad4254f864d1428c12e64841edc43447a70a790ea66997fbf9d78f1314ccd62 WatchSource:0}: Error finding container 9ad4254f864d1428c12e64841edc43447a70a790ea66997fbf9d78f1314ccd62: Status 404 returned error can't find the container with id 9ad4254f864d1428c12e64841edc43447a70a790ea66997fbf9d78f1314ccd62 Jun 11 16:19:00.831086 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:00.831042 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6bbbf84858-z99qw" podStartSLOduration=1.536416964 podStartE2EDuration="5.831029439s" totalImagesPullingTime="4.294612475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:18:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:19:00.8296939 +0000 UTC m=+616.663806467" watchObservedRunningTime="2026-06-11 16:19:00.831029439 +0000 UTC m=+616.665141977" Jun 11 16:19:00.844910 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:00.844867 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-6dc94f689f-rpwx9" podStartSLOduration=0.886015444 podStartE2EDuration="4.844847954s" totalImagesPullingTime="3.95883251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:18:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:19:00.844045252 +0000 UTC m=+616.678157784" watchObservedRunningTime="2026-06-11 16:19:00.844847954 +0000 UTC m=+616.678960511" Jun 11 16:19:01.823682 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:01.823647 2573 generic.go:356] "Generic (PLEG): container finished" podID="51d361a5-3bbe-4910-8e53-79fc4dc35153" containerID="7d17c7773cbf434a46bca541a2d1349a2dd47efff90ff973ea2de58a385b250c" exitCode=0 Jun 11 16:19:01.824169 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:01.823736 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" event={"ID":"51d361a5-3bbe-4910-8e53-79fc4dc35153","Type":"ContainerDied","Data":"7d17c7773cbf434a46bca541a2d1349a2dd47efff90ff973ea2de58a385b250c"} Jun 11 16:19:01.824169 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:01.823797 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" event={"ID":"51d361a5-3bbe-4910-8e53-79fc4dc35153","Type":"ContainerStarted","Data":"9ad4254f864d1428c12e64841edc43447a70a790ea66997fbf9d78f1314ccd62"} Jun 11 16:19:03.834813 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:03.834454 2573 generic.go:356] "Generic (PLEG): container finished" podID="51d361a5-3bbe-4910-8e53-79fc4dc35153" containerID="6dc5bdf577c730a517baa3ca9ce85211aeeca2a182e35ec99b977f2b8c291ccf" exitCode=0 Jun 11 16:19:03.834813 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:03.834642 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" event={"ID":"51d361a5-3bbe-4910-8e53-79fc4dc35153","Type":"ContainerDied","Data":"6dc5bdf577c730a517baa3ca9ce85211aeeca2a182e35ec99b977f2b8c291ccf"} Jun 11 16:19:03.836826 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:03.836799 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-xmkq6" event={"ID":"8ce98183-5dc0-4034-ab51-718722ff4da0","Type":"ContainerStarted","Data":"5f91e5aea7fa3982a502a95c5d00f9884cef5ea243babd9e29ecaad257cd596b"} Jun 11 16:19:03.837000 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:03.836966 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-5dd488fc48-xmkq6" Jun 11 16:19:03.889181 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:03.889133 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-5dd488fc48-xmkq6" podStartSLOduration=4.896311891 podStartE2EDuration="6.88911717s" totalImagesPullingTime="1.992805279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:19:03.887057535 +0000 UTC m=+619.721170089" watchObservedRunningTime="2026-06-11 16:19:03.88911717 +0000 UTC m=+619.723229759" Jun 11 16:19:04.843669 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:04.843634 2573 generic.go:356] "Generic (PLEG): container finished" podID="51d361a5-3bbe-4910-8e53-79fc4dc35153" containerID="b3b2187051bf4a18450219eeadcfd180b805430077be11000bac8d30a6c14ff7" exitCode=0 Jun 11 16:19:04.844146 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:04.843742 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" event={"ID":"51d361a5-3bbe-4910-8e53-79fc4dc35153","Type":"ContainerDied","Data":"b3b2187051bf4a18450219eeadcfd180b805430077be11000bac8d30a6c14ff7"} Jun 11 16:19:05.975100 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:05.975067 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" Jun 11 16:19:06.054272 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.054240 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51d361a5-3bbe-4910-8e53-79fc4dc35153-bundle\") pod \"51d361a5-3bbe-4910-8e53-79fc4dc35153\" (UID: \"51d361a5-3bbe-4910-8e53-79fc4dc35153\") " Jun 11 16:19:06.054432 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.054285 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jq744\" (UniqueName: \"kubernetes.io/projected/51d361a5-3bbe-4910-8e53-79fc4dc35153-kube-api-access-jq744\") pod \"51d361a5-3bbe-4910-8e53-79fc4dc35153\" (UID: \"51d361a5-3bbe-4910-8e53-79fc4dc35153\") " Jun 11 16:19:06.054432 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.054316 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51d361a5-3bbe-4910-8e53-79fc4dc35153-util\") pod \"51d361a5-3bbe-4910-8e53-79fc4dc35153\" (UID: \"51d361a5-3bbe-4910-8e53-79fc4dc35153\") " Jun 11 16:19:06.055190 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.055164 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51d361a5-3bbe-4910-8e53-79fc4dc35153-bundle" (OuterVolumeSpecName: "bundle") pod "51d361a5-3bbe-4910-8e53-79fc4dc35153" (UID: "51d361a5-3bbe-4910-8e53-79fc4dc35153"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:19:06.056626 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.056599 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51d361a5-3bbe-4910-8e53-79fc4dc35153-kube-api-access-jq744" (OuterVolumeSpecName: "kube-api-access-jq744") pod "51d361a5-3bbe-4910-8e53-79fc4dc35153" (UID: "51d361a5-3bbe-4910-8e53-79fc4dc35153"). InnerVolumeSpecName "kube-api-access-jq744". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:19:06.060053 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.060028 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51d361a5-3bbe-4910-8e53-79fc4dc35153-util" (OuterVolumeSpecName: "util") pod "51d361a5-3bbe-4910-8e53-79fc4dc35153" (UID: "51d361a5-3bbe-4910-8e53-79fc4dc35153"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:19:06.155088 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.155053 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51d361a5-3bbe-4910-8e53-79fc4dc35153-bundle\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:19:06.155088 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.155089 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jq744\" (UniqueName: \"kubernetes.io/projected/51d361a5-3bbe-4910-8e53-79fc4dc35153-kube-api-access-jq744\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:19:06.155294 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.155104 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51d361a5-3bbe-4910-8e53-79fc4dc35153-util\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:19:06.241613 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.241573 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-zvhgv"] Jun 11 16:19:06.241971 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.241955 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51d361a5-3bbe-4910-8e53-79fc4dc35153" containerName="pull" Jun 11 16:19:06.242067 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.241974 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="51d361a5-3bbe-4910-8e53-79fc4dc35153" containerName="pull" Jun 11 16:19:06.242067 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.241993 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51d361a5-3bbe-4910-8e53-79fc4dc35153" containerName="extract" Jun 11 16:19:06.242067 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.242002 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="51d361a5-3bbe-4910-8e53-79fc4dc35153" containerName="extract" Jun 11 16:19:06.242067 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.242020 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51d361a5-3bbe-4910-8e53-79fc4dc35153" containerName="util" Jun 11 16:19:06.242067 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.242028 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="51d361a5-3bbe-4910-8e53-79fc4dc35153" containerName="util" Jun 11 16:19:06.242299 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.242127 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="51d361a5-3bbe-4910-8e53-79fc4dc35153" containerName="extract" Jun 11 16:19:06.245229 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.245209 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-zvhgv" Jun 11 16:19:06.249188 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.249157 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-hrp2d\"" Jun 11 16:19:06.249284 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.249166 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 11 16:19:06.271281 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.271258 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-zvhgv"] Jun 11 16:19:06.272822 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.272802 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-bm6p8"] Jun 11 16:19:06.275830 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.275813 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-bm6p8" Jun 11 16:19:06.279055 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.279037 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-d6jrb\"" Jun 11 16:19:06.279168 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.279099 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jun 11 16:19:06.313286 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.313258 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-bm6p8"] Jun 11 16:19:06.356353 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.356323 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mcwg\" (UniqueName: \"kubernetes.io/projected/9c3b9f2f-da71-46da-aa84-78734ead79eb-kube-api-access-4mcwg\") pod \"kserve-controller-manager-69fc866d76-zvhgv\" (UID: \"9c3b9f2f-da71-46da-aa84-78734ead79eb\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zvhgv" Jun 11 16:19:06.356468 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.356366 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/865d7acf-b0a5-479d-929a-aae71cecab82-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-bm6p8\" (UID: \"865d7acf-b0a5-479d-929a-aae71cecab82\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-bm6p8" Jun 11 16:19:06.356468 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.356395 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btdvf\" (UniqueName: \"kubernetes.io/projected/865d7acf-b0a5-479d-929a-aae71cecab82-kube-api-access-btdvf\") pod \"llmisvc-controller-manager-864bfd7bc9-bm6p8\" (UID: \"865d7acf-b0a5-479d-929a-aae71cecab82\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-bm6p8" Jun 11 16:19:06.356468 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.356441 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c3b9f2f-da71-46da-aa84-78734ead79eb-cert\") pod \"kserve-controller-manager-69fc866d76-zvhgv\" (UID: \"9c3b9f2f-da71-46da-aa84-78734ead79eb\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zvhgv" Jun 11 16:19:06.457196 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.457100 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mcwg\" (UniqueName: \"kubernetes.io/projected/9c3b9f2f-da71-46da-aa84-78734ead79eb-kube-api-access-4mcwg\") pod \"kserve-controller-manager-69fc866d76-zvhgv\" (UID: \"9c3b9f2f-da71-46da-aa84-78734ead79eb\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zvhgv" Jun 11 16:19:06.457196 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.457162 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/865d7acf-b0a5-479d-929a-aae71cecab82-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-bm6p8\" (UID: \"865d7acf-b0a5-479d-929a-aae71cecab82\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-bm6p8" Jun 11 16:19:06.457196 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.457192 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btdvf\" (UniqueName: \"kubernetes.io/projected/865d7acf-b0a5-479d-929a-aae71cecab82-kube-api-access-btdvf\") pod \"llmisvc-controller-manager-864bfd7bc9-bm6p8\" (UID: \"865d7acf-b0a5-479d-929a-aae71cecab82\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-bm6p8" Jun 11 16:19:06.457470 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.457227 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c3b9f2f-da71-46da-aa84-78734ead79eb-cert\") pod \"kserve-controller-manager-69fc866d76-zvhgv\" (UID: \"9c3b9f2f-da71-46da-aa84-78734ead79eb\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zvhgv" Jun 11 16:19:06.457470 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:19:06.457344 2573 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jun 11 16:19:06.457470 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:19:06.457410 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c3b9f2f-da71-46da-aa84-78734ead79eb-cert podName:9c3b9f2f-da71-46da-aa84-78734ead79eb nodeName:}" failed. No retries permitted until 2026-06-11 16:19:06.957389158 +0000 UTC m=+622.791501678 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9c3b9f2f-da71-46da-aa84-78734ead79eb-cert") pod "kserve-controller-manager-69fc866d76-zvhgv" (UID: "9c3b9f2f-da71-46da-aa84-78734ead79eb") : secret "kserve-webhook-server-cert" not found Jun 11 16:19:06.459749 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.459721 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/865d7acf-b0a5-479d-929a-aae71cecab82-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-bm6p8\" (UID: \"865d7acf-b0a5-479d-929a-aae71cecab82\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-bm6p8" Jun 11 16:19:06.472751 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.472726 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mcwg\" (UniqueName: \"kubernetes.io/projected/9c3b9f2f-da71-46da-aa84-78734ead79eb-kube-api-access-4mcwg\") pod \"kserve-controller-manager-69fc866d76-zvhgv\" (UID: \"9c3b9f2f-da71-46da-aa84-78734ead79eb\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zvhgv" Jun 11 16:19:06.480220 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.480199 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btdvf\" (UniqueName: \"kubernetes.io/projected/865d7acf-b0a5-479d-929a-aae71cecab82-kube-api-access-btdvf\") pod \"llmisvc-controller-manager-864bfd7bc9-bm6p8\" (UID: \"865d7acf-b0a5-479d-929a-aae71cecab82\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-bm6p8" Jun 11 16:19:06.585626 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.585593 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-bm6p8" Jun 11 16:19:06.723379 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.723354 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-bm6p8"] Jun 11 16:19:06.724784 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:19:06.724756 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod865d7acf_b0a5_479d_929a_aae71cecab82.slice/crio-70d472c7fade1e467b9f8e37ff86f6de0bfaae0d19fdf29e947c79e32a365187 WatchSource:0}: Error finding container 70d472c7fade1e467b9f8e37ff86f6de0bfaae0d19fdf29e947c79e32a365187: Status 404 returned error can't find the container with id 70d472c7fade1e467b9f8e37ff86f6de0bfaae0d19fdf29e947c79e32a365187 Jun 11 16:19:06.853103 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.853056 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-bm6p8" event={"ID":"865d7acf-b0a5-479d-929a-aae71cecab82","Type":"ContainerStarted","Data":"70d472c7fade1e467b9f8e37ff86f6de0bfaae0d19fdf29e947c79e32a365187"} Jun 11 16:19:06.855004 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.854985 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" Jun 11 16:19:06.855113 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.854984 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06wrqxs" event={"ID":"51d361a5-3bbe-4910-8e53-79fc4dc35153","Type":"ContainerDied","Data":"9ad4254f864d1428c12e64841edc43447a70a790ea66997fbf9d78f1314ccd62"} Jun 11 16:19:06.855113 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.855105 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9ad4254f864d1428c12e64841edc43447a70a790ea66997fbf9d78f1314ccd62" Jun 11 16:19:06.961901 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.961865 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c3b9f2f-da71-46da-aa84-78734ead79eb-cert\") pod \"kserve-controller-manager-69fc866d76-zvhgv\" (UID: \"9c3b9f2f-da71-46da-aa84-78734ead79eb\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zvhgv" Jun 11 16:19:06.964480 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:06.964449 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9c3b9f2f-da71-46da-aa84-78734ead79eb-cert\") pod \"kserve-controller-manager-69fc866d76-zvhgv\" (UID: \"9c3b9f2f-da71-46da-aa84-78734ead79eb\") " pod="opendatahub/kserve-controller-manager-69fc866d76-zvhgv" Jun 11 16:19:07.154973 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:07.154938 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-zvhgv" Jun 11 16:19:07.304106 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:07.304082 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-zvhgv"] Jun 11 16:19:07.305889 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:19:07.305864 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c3b9f2f_da71_46da_aa84_78734ead79eb.slice/crio-5b3bcad1a3157beb2c7ba8a4fb9a2acf77491befc585f06bb92dff034142e80d WatchSource:0}: Error finding container 5b3bcad1a3157beb2c7ba8a4fb9a2acf77491befc585f06bb92dff034142e80d: Status 404 returned error can't find the container with id 5b3bcad1a3157beb2c7ba8a4fb9a2acf77491befc585f06bb92dff034142e80d Jun 11 16:19:07.861710 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:07.861678 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-zvhgv" event={"ID":"9c3b9f2f-da71-46da-aa84-78734ead79eb","Type":"ContainerStarted","Data":"5b3bcad1a3157beb2c7ba8a4fb9a2acf77491befc585f06bb92dff034142e80d"} Jun 11 16:19:11.826101 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:11.826069 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-6dc94f689f-rpwx9" Jun 11 16:19:11.886601 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:11.886566 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-zvhgv" event={"ID":"9c3b9f2f-da71-46da-aa84-78734ead79eb","Type":"ContainerStarted","Data":"12a296922a14c567afbfaf4325b3137ae3369d535dae96d7cd6edb5441193434"} Jun 11 16:19:11.886766 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:11.886648 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-69fc866d76-zvhgv" Jun 11 16:19:11.888425 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:11.888385 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-bm6p8" event={"ID":"865d7acf-b0a5-479d-929a-aae71cecab82","Type":"ContainerStarted","Data":"5b83d23a633feea932e4d41b28ada0cce7f97c27becac2cd4b5ac5a6586e2b54"} Jun 11 16:19:11.888559 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:11.888511 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-bm6p8" Jun 11 16:19:11.903659 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:11.903616 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-69fc866d76-zvhgv" podStartSLOduration=2.09379827 podStartE2EDuration="5.903601686s" totalImagesPullingTime="3.809803416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:19:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:19:11.902298761 +0000 UTC m=+627.736411302" watchObservedRunningTime="2026-06-11 16:19:11.903601686 +0000 UTC m=+627.737714225" Jun 11 16:19:11.919937 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:11.919883 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-bm6p8" podStartSLOduration=1.52371172 podStartE2EDuration="5.919868065s" totalImagesPullingTime="4.396156345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:19:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:19:11.918677656 +0000 UTC m=+627.752790195" watchObservedRunningTime="2026-06-11 16:19:11.919868065 +0000 UTC m=+627.753980631" Jun 11 16:19:14.847809 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:14.847783 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-5dd488fc48-xmkq6" Jun 11 16:19:15.839408 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:15.839373 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk"] Jun 11 16:19:15.842889 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:15.842872 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" Jun 11 16:19:15.851597 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:15.851579 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qh6np\"" Jun 11 16:19:15.851909 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:15.851732 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 11 16:19:15.851976 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:15.851945 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 11 16:19:15.944120 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:15.944087 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05e132ac-4e35-4e2d-a594-c6e27809b076-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk\" (UID: \"05e132ac-4e35-4e2d-a594-c6e27809b076\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" Jun 11 16:19:15.944120 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:15.944123 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwclw\" (UniqueName: \"kubernetes.io/projected/05e132ac-4e35-4e2d-a594-c6e27809b076-kube-api-access-gwclw\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk\" (UID: \"05e132ac-4e35-4e2d-a594-c6e27809b076\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" Jun 11 16:19:15.944342 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:15.944192 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05e132ac-4e35-4e2d-a594-c6e27809b076-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk\" (UID: \"05e132ac-4e35-4e2d-a594-c6e27809b076\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" Jun 11 16:19:16.019152 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:16.019116 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk"] Jun 11 16:19:16.045040 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:16.045012 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05e132ac-4e35-4e2d-a594-c6e27809b076-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk\" (UID: \"05e132ac-4e35-4e2d-a594-c6e27809b076\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" Jun 11 16:19:16.045209 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:16.045049 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwclw\" (UniqueName: \"kubernetes.io/projected/05e132ac-4e35-4e2d-a594-c6e27809b076-kube-api-access-gwclw\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk\" (UID: \"05e132ac-4e35-4e2d-a594-c6e27809b076\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" Jun 11 16:19:16.045209 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:16.045087 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05e132ac-4e35-4e2d-a594-c6e27809b076-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk\" (UID: \"05e132ac-4e35-4e2d-a594-c6e27809b076\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" Jun 11 16:19:16.045425 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:16.045402 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05e132ac-4e35-4e2d-a594-c6e27809b076-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk\" (UID: \"05e132ac-4e35-4e2d-a594-c6e27809b076\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" Jun 11 16:19:16.045487 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:16.045432 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05e132ac-4e35-4e2d-a594-c6e27809b076-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk\" (UID: \"05e132ac-4e35-4e2d-a594-c6e27809b076\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" Jun 11 16:19:16.091640 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:16.091566 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwclw\" (UniqueName: \"kubernetes.io/projected/05e132ac-4e35-4e2d-a594-c6e27809b076-kube-api-access-gwclw\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk\" (UID: \"05e132ac-4e35-4e2d-a594-c6e27809b076\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" Jun 11 16:19:16.151625 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:16.151591 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" Jun 11 16:19:16.333395 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:16.333364 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk"] Jun 11 16:19:16.334106 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:19:16.334083 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05e132ac_4e35_4e2d_a594_c6e27809b076.slice/crio-99c05f29338ca44c256e0ad32bcfa436090f38be0161365249280f99ba34c03f WatchSource:0}: Error finding container 99c05f29338ca44c256e0ad32bcfa436090f38be0161365249280f99ba34c03f: Status 404 returned error can't find the container with id 99c05f29338ca44c256e0ad32bcfa436090f38be0161365249280f99ba34c03f Jun 11 16:19:16.916239 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:16.916205 2573 generic.go:356] "Generic (PLEG): container finished" podID="05e132ac-4e35-4e2d-a594-c6e27809b076" containerID="b9993287b75c91872697455e2e5ae3b94b1fb0477d337e72e8517ee5b5be065c" exitCode=0 Jun 11 16:19:16.916650 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:16.916294 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" event={"ID":"05e132ac-4e35-4e2d-a594-c6e27809b076","Type":"ContainerDied","Data":"b9993287b75c91872697455e2e5ae3b94b1fb0477d337e72e8517ee5b5be065c"} Jun 11 16:19:16.916650 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:16.916355 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" event={"ID":"05e132ac-4e35-4e2d-a594-c6e27809b076","Type":"ContainerStarted","Data":"99c05f29338ca44c256e0ad32bcfa436090f38be0161365249280f99ba34c03f"} Jun 11 16:19:17.923102 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:17.923055 2573 generic.go:356] "Generic (PLEG): container finished" podID="05e132ac-4e35-4e2d-a594-c6e27809b076" containerID="edd4e3bd0f8046ad9e3ef09cba414b635271138bde03d0ba82cb30c80154527a" exitCode=0 Jun 11 16:19:17.923102 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:17.923100 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" event={"ID":"05e132ac-4e35-4e2d-a594-c6e27809b076","Type":"ContainerDied","Data":"edd4e3bd0f8046ad9e3ef09cba414b635271138bde03d0ba82cb30c80154527a"} Jun 11 16:19:17.962402 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:17.962379 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-9j87l"] Jun 11 16:19:17.966895 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:17.966878 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-9j87l" Jun 11 16:19:17.970519 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:17.970501 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-dpqrr\"" Jun 11 16:19:17.970827 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:17.970808 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Jun 11 16:19:17.970958 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:17.970888 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Jun 11 16:19:18.000301 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:18.000271 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-9j87l"] Jun 11 16:19:18.061560 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:18.061525 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/bcb7bdd4-4abf-47f3-b4a1-3cc7134db49a-operator-config\") pod \"servicemesh-operator3-57f65f65fb-9j87l\" (UID: \"bcb7bdd4-4abf-47f3-b4a1-3cc7134db49a\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-9j87l" Jun 11 16:19:18.061560 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:18.061564 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shkkj\" (UniqueName: \"kubernetes.io/projected/bcb7bdd4-4abf-47f3-b4a1-3cc7134db49a-kube-api-access-shkkj\") pod \"servicemesh-operator3-57f65f65fb-9j87l\" (UID: \"bcb7bdd4-4abf-47f3-b4a1-3cc7134db49a\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-9j87l" Jun 11 16:19:18.162252 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:18.162220 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/bcb7bdd4-4abf-47f3-b4a1-3cc7134db49a-operator-config\") pod \"servicemesh-operator3-57f65f65fb-9j87l\" (UID: \"bcb7bdd4-4abf-47f3-b4a1-3cc7134db49a\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-9j87l" Jun 11 16:19:18.162438 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:18.162261 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shkkj\" (UniqueName: \"kubernetes.io/projected/bcb7bdd4-4abf-47f3-b4a1-3cc7134db49a-kube-api-access-shkkj\") pod \"servicemesh-operator3-57f65f65fb-9j87l\" (UID: \"bcb7bdd4-4abf-47f3-b4a1-3cc7134db49a\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-9j87l" Jun 11 16:19:18.165296 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:18.165267 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/bcb7bdd4-4abf-47f3-b4a1-3cc7134db49a-operator-config\") pod \"servicemesh-operator3-57f65f65fb-9j87l\" (UID: \"bcb7bdd4-4abf-47f3-b4a1-3cc7134db49a\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-9j87l" Jun 11 16:19:18.175755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:18.175696 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shkkj\" (UniqueName: \"kubernetes.io/projected/bcb7bdd4-4abf-47f3-b4a1-3cc7134db49a-kube-api-access-shkkj\") pod \"servicemesh-operator3-57f65f65fb-9j87l\" (UID: \"bcb7bdd4-4abf-47f3-b4a1-3cc7134db49a\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-9j87l" Jun 11 16:19:18.275545 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:18.275482 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-9j87l" Jun 11 16:19:18.443295 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:18.443271 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-9j87l"] Jun 11 16:19:18.445439 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:19:18.445421 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb7bdd4_4abf_47f3_b4a1_3cc7134db49a.slice/crio-76b6ab70e1052e26677a355a8afc97fca99332874254d707b33d735d39711ab6 WatchSource:0}: Error finding container 76b6ab70e1052e26677a355a8afc97fca99332874254d707b33d735d39711ab6: Status 404 returned error can't find the container with id 76b6ab70e1052e26677a355a8afc97fca99332874254d707b33d735d39711ab6 Jun 11 16:19:18.929639 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:18.929606 2573 generic.go:356] "Generic (PLEG): container finished" podID="05e132ac-4e35-4e2d-a594-c6e27809b076" containerID="4c3377cb199c7dba8b9418de5fc17e9e43bc60642ec77a03fc5544cd88903a3a" exitCode=0 Jun 11 16:19:18.930045 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:18.929684 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" event={"ID":"05e132ac-4e35-4e2d-a594-c6e27809b076","Type":"ContainerDied","Data":"4c3377cb199c7dba8b9418de5fc17e9e43bc60642ec77a03fc5544cd88903a3a"} Jun 11 16:19:18.931235 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:18.931218 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-9j87l" event={"ID":"bcb7bdd4-4abf-47f3-b4a1-3cc7134db49a","Type":"ContainerStarted","Data":"76b6ab70e1052e26677a355a8afc97fca99332874254d707b33d735d39711ab6"} Jun 11 16:19:20.074318 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:20.074292 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" Jun 11 16:19:20.179070 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:20.179040 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwclw\" (UniqueName: \"kubernetes.io/projected/05e132ac-4e35-4e2d-a594-c6e27809b076-kube-api-access-gwclw\") pod \"05e132ac-4e35-4e2d-a594-c6e27809b076\" (UID: \"05e132ac-4e35-4e2d-a594-c6e27809b076\") " Jun 11 16:19:20.179232 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:20.179091 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05e132ac-4e35-4e2d-a594-c6e27809b076-bundle\") pod \"05e132ac-4e35-4e2d-a594-c6e27809b076\" (UID: \"05e132ac-4e35-4e2d-a594-c6e27809b076\") " Jun 11 16:19:20.179232 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:20.179123 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05e132ac-4e35-4e2d-a594-c6e27809b076-util\") pod \"05e132ac-4e35-4e2d-a594-c6e27809b076\" (UID: \"05e132ac-4e35-4e2d-a594-c6e27809b076\") " Jun 11 16:19:20.180352 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:20.180319 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05e132ac-4e35-4e2d-a594-c6e27809b076-bundle" (OuterVolumeSpecName: "bundle") pod "05e132ac-4e35-4e2d-a594-c6e27809b076" (UID: "05e132ac-4e35-4e2d-a594-c6e27809b076"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:19:20.181155 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:20.181127 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05e132ac-4e35-4e2d-a594-c6e27809b076-kube-api-access-gwclw" (OuterVolumeSpecName: "kube-api-access-gwclw") pod "05e132ac-4e35-4e2d-a594-c6e27809b076" (UID: "05e132ac-4e35-4e2d-a594-c6e27809b076"). InnerVolumeSpecName "kube-api-access-gwclw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:19:20.187700 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:20.187655 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05e132ac-4e35-4e2d-a594-c6e27809b076-util" (OuterVolumeSpecName: "util") pod "05e132ac-4e35-4e2d-a594-c6e27809b076" (UID: "05e132ac-4e35-4e2d-a594-c6e27809b076"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:19:20.279908 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:20.279823 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwclw\" (UniqueName: \"kubernetes.io/projected/05e132ac-4e35-4e2d-a594-c6e27809b076-kube-api-access-gwclw\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:19:20.279908 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:20.279858 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05e132ac-4e35-4e2d-a594-c6e27809b076-bundle\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:19:20.279908 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:20.279872 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05e132ac-4e35-4e2d-a594-c6e27809b076-util\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:19:20.945216 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:20.945179 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" event={"ID":"05e132ac-4e35-4e2d-a594-c6e27809b076","Type":"ContainerDied","Data":"99c05f29338ca44c256e0ad32bcfa436090f38be0161365249280f99ba34c03f"} Jun 11 16:19:20.945216 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:20.945213 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762azhpgk" Jun 11 16:19:20.945424 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:20.945231 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99c05f29338ca44c256e0ad32bcfa436090f38be0161365249280f99ba34c03f" Jun 11 16:19:21.952565 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:21.952531 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-9j87l" event={"ID":"bcb7bdd4-4abf-47f3-b4a1-3cc7134db49a","Type":"ContainerStarted","Data":"f8c08d598aa6def73a21e9ce910c4f0c7dbacff00daf2177b0aa1c09dae7f61e"} Jun 11 16:19:21.952960 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:21.952650 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-9j87l" Jun 11 16:19:21.982309 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:21.982049 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-57f65f65fb-9j87l" podStartSLOduration=2.295328154 podStartE2EDuration="4.982032381s" totalImagesPullingTime="2.686704227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:19:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:19:21.980608511 +0000 UTC m=+637.814721050" watchObservedRunningTime="2026-06-11 16:19:21.982032381 +0000 UTC m=+637.816144920" Jun 11 16:19:26.258445 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.258410 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-758d6cf95f-nkl4c"] Jun 11 16:19:26.258961 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.258942 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05e132ac-4e35-4e2d-a594-c6e27809b076" containerName="util" Jun 11 16:19:26.259056 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.258963 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="05e132ac-4e35-4e2d-a594-c6e27809b076" containerName="util" Jun 11 16:19:26.259056 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.258991 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05e132ac-4e35-4e2d-a594-c6e27809b076" containerName="extract" Jun 11 16:19:26.259056 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.259000 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="05e132ac-4e35-4e2d-a594-c6e27809b076" containerName="extract" Jun 11 16:19:26.259056 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.259015 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05e132ac-4e35-4e2d-a594-c6e27809b076" containerName="pull" Jun 11 16:19:26.259056 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.259024 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="05e132ac-4e35-4e2d-a594-c6e27809b076" containerName="pull" Jun 11 16:19:26.259282 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.259133 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="05e132ac-4e35-4e2d-a594-c6e27809b076" containerName="extract" Jun 11 16:19:26.262523 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.262500 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-758d6cf95f-nkl4c" Jun 11 16:19:26.265095 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.265076 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-pdf85\"" Jun 11 16:19:26.265362 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.265344 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 11 16:19:26.276497 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.276462 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-758d6cf95f-nkl4c"] Jun 11 16:19:26.325071 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.325042 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/82b0d29d-8a3e-4840-b049-19fa76dcb173-redirect-config\") pod \"dashboard-redirect-758d6cf95f-nkl4c\" (UID: \"82b0d29d-8a3e-4840-b049-19fa76dcb173\") " pod="opendatahub/dashboard-redirect-758d6cf95f-nkl4c" Jun 11 16:19:26.325211 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.325075 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wlgt\" (UniqueName: \"kubernetes.io/projected/82b0d29d-8a3e-4840-b049-19fa76dcb173-kube-api-access-8wlgt\") pod \"dashboard-redirect-758d6cf95f-nkl4c\" (UID: \"82b0d29d-8a3e-4840-b049-19fa76dcb173\") " pod="opendatahub/dashboard-redirect-758d6cf95f-nkl4c" Jun 11 16:19:26.426233 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.426199 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/82b0d29d-8a3e-4840-b049-19fa76dcb173-redirect-config\") pod \"dashboard-redirect-758d6cf95f-nkl4c\" (UID: \"82b0d29d-8a3e-4840-b049-19fa76dcb173\") " pod="opendatahub/dashboard-redirect-758d6cf95f-nkl4c" Jun 11 16:19:26.426233 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.426232 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wlgt\" (UniqueName: \"kubernetes.io/projected/82b0d29d-8a3e-4840-b049-19fa76dcb173-kube-api-access-8wlgt\") pod \"dashboard-redirect-758d6cf95f-nkl4c\" (UID: \"82b0d29d-8a3e-4840-b049-19fa76dcb173\") " pod="opendatahub/dashboard-redirect-758d6cf95f-nkl4c" Jun 11 16:19:26.426916 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.426898 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/82b0d29d-8a3e-4840-b049-19fa76dcb173-redirect-config\") pod \"dashboard-redirect-758d6cf95f-nkl4c\" (UID: \"82b0d29d-8a3e-4840-b049-19fa76dcb173\") " pod="opendatahub/dashboard-redirect-758d6cf95f-nkl4c" Jun 11 16:19:26.434679 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.434653 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wlgt\" (UniqueName: \"kubernetes.io/projected/82b0d29d-8a3e-4840-b049-19fa76dcb173-kube-api-access-8wlgt\") pod \"dashboard-redirect-758d6cf95f-nkl4c\" (UID: \"82b0d29d-8a3e-4840-b049-19fa76dcb173\") " pod="opendatahub/dashboard-redirect-758d6cf95f-nkl4c" Jun 11 16:19:26.572466 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.572379 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-758d6cf95f-nkl4c" Jun 11 16:19:26.706993 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.706961 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-758d6cf95f-nkl4c"] Jun 11 16:19:26.710409 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:19:26.710374 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82b0d29d_8a3e_4840_b049_19fa76dcb173.slice/crio-ad0f2e679fa4dee13e962be3ace98763e0c94f0f43adf8341690c1b008a2e5df WatchSource:0}: Error finding container ad0f2e679fa4dee13e962be3ace98763e0c94f0f43adf8341690c1b008a2e5df: Status 404 returned error can't find the container with id ad0f2e679fa4dee13e962be3ace98763e0c94f0f43adf8341690c1b008a2e5df Jun 11 16:19:26.977246 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:26.977209 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-758d6cf95f-nkl4c" event={"ID":"82b0d29d-8a3e-4840-b049-19fa76dcb173","Type":"ContainerStarted","Data":"ad0f2e679fa4dee13e962be3ace98763e0c94f0f43adf8341690c1b008a2e5df"} Jun 11 16:19:31.000324 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:31.000293 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-758d6cf95f-nkl4c" event={"ID":"82b0d29d-8a3e-4840-b049-19fa76dcb173","Type":"ContainerStarted","Data":"fea8e08d8d82102d9badb6141f742a44298eae1ae1cd15196635ce905c67c130"} Jun 11 16:19:31.020708 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:31.020655 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-758d6cf95f-nkl4c" podStartSLOduration=0.815130989 podStartE2EDuration="5.02063879s" totalImagesPullingTime="4.205507801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:19:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:19:31.018316118 +0000 UTC m=+646.852428656" watchObservedRunningTime="2026-06-11 16:19:31.02063879 +0000 UTC m=+646.854751373" Jun 11 16:19:32.960306 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:32.960275 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-9j87l" Jun 11 16:19:42.895481 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:42.895441 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-bm6p8" Jun 11 16:19:42.898736 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:42.898714 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-69fc866d76-zvhgv" Jun 11 16:19:47.855138 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:47.855105 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22"] Jun 11 16:19:47.858504 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:47.858482 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:47.861053 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:47.861033 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 11 16:19:47.861516 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:47.861497 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 11 16:19:47.861662 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:47.861646 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-s9vr7\"" Jun 11 16:19:47.861735 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:47.861709 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 11 16:19:47.861951 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:47.861937 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 11 16:19:47.875481 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:47.875458 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22"] Jun 11 16:19:47.900617 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:47.900589 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:47.900756 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:47.900646 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:47.900756 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:47.900703 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:47.900756 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:47.900733 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:47.900884 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:47.900791 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:47.900884 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:47.900841 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:47.900981 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:47.900885 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2zd7\" (UniqueName: \"kubernetes.io/projected/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-kube-api-access-p2zd7\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:48.001616 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:48.001571 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:48.001616 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:48.001620 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:48.001856 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:48.001642 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:48.001856 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:48.001698 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:48.001856 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:48.001723 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:48.001856 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:48.001761 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2zd7\" (UniqueName: \"kubernetes.io/projected/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-kube-api-access-p2zd7\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:48.001856 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:48.001798 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:48.002421 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:48.002395 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:48.004387 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:48.004358 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:48.004489 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:48.004444 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:48.004812 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:48.004791 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:48.004858 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:48.004829 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:48.009629 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:48.009606 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:48.010155 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:48.010133 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2zd7\" (UniqueName: \"kubernetes.io/projected/e24ed10d-33f8-4bcb-8609-7c04ee8dcf43-kube-api-access-p2zd7\") pod \"istiod-openshift-gateway-75c67f8887-dns22\" (UID: \"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:48.167381 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:48.167352 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:48.315461 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:48.311658 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22"] Jun 11 16:19:48.316887 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:19:48.316859 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode24ed10d_33f8_4bcb_8609_7c04ee8dcf43.slice/crio-7e2975eed84425b701937b9dee627de3d4c1c6adce8fae9b8f37fa76f6a08283 WatchSource:0}: Error finding container 7e2975eed84425b701937b9dee627de3d4c1c6adce8fae9b8f37fa76f6a08283: Status 404 returned error can't find the container with id 7e2975eed84425b701937b9dee627de3d4c1c6adce8fae9b8f37fa76f6a08283 Jun 11 16:19:49.088899 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:49.088854 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" event={"ID":"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43","Type":"ContainerStarted","Data":"7e2975eed84425b701937b9dee627de3d4c1c6adce8fae9b8f37fa76f6a08283"} Jun 11 16:19:50.738634 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:50.738577 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 11 16:19:50.738904 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:50.738678 2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jun 11 16:19:51.101813 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:51.101721 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" event={"ID":"e24ed10d-33f8-4bcb-8609-7c04ee8dcf43","Type":"ContainerStarted","Data":"7105bb9bb0d4dd7aaeb1477c26890a8e04f2146e4f06b8b70d5d2f8955a22241"} Jun 11 16:19:51.101997 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:51.101880 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:19:51.103692 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:51.103669 2573 patch_prober.go:28] interesting pod/istiod-openshift-gateway-75c67f8887-dns22 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jun 11 16:19:51.103797 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:51.103705 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" podUID="e24ed10d-33f8-4bcb-8609-7c04ee8dcf43" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 11 16:19:51.124032 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:51.123987 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" podStartSLOduration=1.704727208 podStartE2EDuration="4.123972774s" totalImagesPullingTime="2.419245566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:19:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:19:51.121459716 +0000 UTC m=+666.955572256" watchObservedRunningTime="2026-06-11 16:19:51.123972774 +0000 UTC m=+666.958085312" Jun 11 16:19:52.106738 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:19:52.106706 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-dns22" Jun 11 16:20:12.416590 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:12.416552 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc"] Jun 11 16:20:12.420231 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:12.420213 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" Jun 11 16:20:12.422461 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:12.422442 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 11 16:20:12.422722 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:12.422706 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 11 16:20:12.423136 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:12.423122 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tx77h\"" Jun 11 16:20:12.429426 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:12.429391 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc"] Jun 11 16:20:12.522857 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:12.522814 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7366b4af-f894-4c26-8491-e6772f6a723d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc\" (UID: \"7366b4af-f894-4c26-8491-e6772f6a723d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" Jun 11 16:20:12.522857 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:12.522859 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7366b4af-f894-4c26-8491-e6772f6a723d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc\" (UID: \"7366b4af-f894-4c26-8491-e6772f6a723d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" Jun 11 16:20:12.523141 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:12.522894 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdvrw\" (UniqueName: \"kubernetes.io/projected/7366b4af-f894-4c26-8491-e6772f6a723d-kube-api-access-kdvrw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc\" (UID: \"7366b4af-f894-4c26-8491-e6772f6a723d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" Jun 11 16:20:12.624103 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:12.624061 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7366b4af-f894-4c26-8491-e6772f6a723d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc\" (UID: \"7366b4af-f894-4c26-8491-e6772f6a723d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" Jun 11 16:20:12.624103 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:12.624101 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7366b4af-f894-4c26-8491-e6772f6a723d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc\" (UID: \"7366b4af-f894-4c26-8491-e6772f6a723d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" Jun 11 16:20:12.624323 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:12.624130 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdvrw\" (UniqueName: \"kubernetes.io/projected/7366b4af-f894-4c26-8491-e6772f6a723d-kube-api-access-kdvrw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc\" (UID: \"7366b4af-f894-4c26-8491-e6772f6a723d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" Jun 11 16:20:12.624500 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:12.624481 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7366b4af-f894-4c26-8491-e6772f6a723d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc\" (UID: \"7366b4af-f894-4c26-8491-e6772f6a723d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" Jun 11 16:20:12.624536 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:12.624473 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7366b4af-f894-4c26-8491-e6772f6a723d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc\" (UID: \"7366b4af-f894-4c26-8491-e6772f6a723d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" Jun 11 16:20:12.635793 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:12.635761 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdvrw\" (UniqueName: \"kubernetes.io/projected/7366b4af-f894-4c26-8491-e6772f6a723d-kube-api-access-kdvrw\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc\" (UID: \"7366b4af-f894-4c26-8491-e6772f6a723d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" Jun 11 16:20:12.730770 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:12.730679 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" Jun 11 16:20:12.874798 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:12.872448 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc"] Jun 11 16:20:12.876371 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:20:12.876328 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7366b4af_f894_4c26_8491_e6772f6a723d.slice/crio-eecb7bb433fb29c7487d2414efdd3680590831d3c8d9695e9a7a06ddbd629100 WatchSource:0}: Error finding container eecb7bb433fb29c7487d2414efdd3680590831d3c8d9695e9a7a06ddbd629100: Status 404 returned error can't find the container with id eecb7bb433fb29c7487d2414efdd3680590831d3c8d9695e9a7a06ddbd629100 Jun 11 16:20:13.012307 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.012272 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82"] Jun 11 16:20:13.015951 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.015913 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" Jun 11 16:20:13.024047 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.024016 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82"] Jun 11 16:20:13.128413 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.128372 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/071f0967-15ae-4799-a8c3-c31510360054-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82\" (UID: \"071f0967-15ae-4799-a8c3-c31510360054\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" Jun 11 16:20:13.128413 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.128411 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/071f0967-15ae-4799-a8c3-c31510360054-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82\" (UID: \"071f0967-15ae-4799-a8c3-c31510360054\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" Jun 11 16:20:13.128642 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.128446 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxmjz\" (UniqueName: \"kubernetes.io/projected/071f0967-15ae-4799-a8c3-c31510360054-kube-api-access-lxmjz\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82\" (UID: \"071f0967-15ae-4799-a8c3-c31510360054\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" Jun 11 16:20:13.217357 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.217321 2573 generic.go:356] "Generic (PLEG): container finished" podID="7366b4af-f894-4c26-8491-e6772f6a723d" containerID="bf88ddffac8e6231b90a305e9cbc4275a805d8f1292019fee52bae2363c78f4d" exitCode=0 Jun 11 16:20:13.217515 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.217378 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" event={"ID":"7366b4af-f894-4c26-8491-e6772f6a723d","Type":"ContainerDied","Data":"bf88ddffac8e6231b90a305e9cbc4275a805d8f1292019fee52bae2363c78f4d"} Jun 11 16:20:13.217515 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.217418 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" event={"ID":"7366b4af-f894-4c26-8491-e6772f6a723d","Type":"ContainerStarted","Data":"eecb7bb433fb29c7487d2414efdd3680590831d3c8d9695e9a7a06ddbd629100"} Jun 11 16:20:13.229123 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.229093 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/071f0967-15ae-4799-a8c3-c31510360054-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82\" (UID: \"071f0967-15ae-4799-a8c3-c31510360054\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" Jun 11 16:20:13.229301 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.229128 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/071f0967-15ae-4799-a8c3-c31510360054-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82\" (UID: \"071f0967-15ae-4799-a8c3-c31510360054\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" Jun 11 16:20:13.229301 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.229182 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxmjz\" (UniqueName: \"kubernetes.io/projected/071f0967-15ae-4799-a8c3-c31510360054-kube-api-access-lxmjz\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82\" (UID: \"071f0967-15ae-4799-a8c3-c31510360054\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" Jun 11 16:20:13.229573 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.229550 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/071f0967-15ae-4799-a8c3-c31510360054-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82\" (UID: \"071f0967-15ae-4799-a8c3-c31510360054\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" Jun 11 16:20:13.229651 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.229591 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/071f0967-15ae-4799-a8c3-c31510360054-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82\" (UID: \"071f0967-15ae-4799-a8c3-c31510360054\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" Jun 11 16:20:13.241107 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.241034 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxmjz\" (UniqueName: \"kubernetes.io/projected/071f0967-15ae-4799-a8c3-c31510360054-kube-api-access-lxmjz\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82\" (UID: \"071f0967-15ae-4799-a8c3-c31510360054\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" Jun 11 16:20:13.342159 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.342121 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" Jun 11 16:20:13.474688 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.474662 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82"] Jun 11 16:20:13.476965 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:20:13.476903 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod071f0967_15ae_4799_a8c3_c31510360054.slice/crio-6aa5ca67675efec5d26e2b923ff5c281efda1c877ce4b5e2da9e6c89ed45be6f WatchSource:0}: Error finding container 6aa5ca67675efec5d26e2b923ff5c281efda1c877ce4b5e2da9e6c89ed45be6f: Status 404 returned error can't find the container with id 6aa5ca67675efec5d26e2b923ff5c281efda1c877ce4b5e2da9e6c89ed45be6f Jun 11 16:20:13.617889 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.617859 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28"] Jun 11 16:20:13.621591 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.621567 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" Jun 11 16:20:13.629696 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.629677 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28"] Jun 11 16:20:13.734830 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.734795 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghbdn\" (UniqueName: \"kubernetes.io/projected/30a58c66-a2a9-4b16-bbad-b6927c94afb4-kube-api-access-ghbdn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28\" (UID: \"30a58c66-a2a9-4b16-bbad-b6927c94afb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" Jun 11 16:20:13.735187 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.735140 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30a58c66-a2a9-4b16-bbad-b6927c94afb4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28\" (UID: \"30a58c66-a2a9-4b16-bbad-b6927c94afb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" Jun 11 16:20:13.735299 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.735189 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30a58c66-a2a9-4b16-bbad-b6927c94afb4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28\" (UID: \"30a58c66-a2a9-4b16-bbad-b6927c94afb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" Jun 11 16:20:13.836144 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.836067 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghbdn\" (UniqueName: \"kubernetes.io/projected/30a58c66-a2a9-4b16-bbad-b6927c94afb4-kube-api-access-ghbdn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28\" (UID: \"30a58c66-a2a9-4b16-bbad-b6927c94afb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" Jun 11 16:20:13.836299 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.836143 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30a58c66-a2a9-4b16-bbad-b6927c94afb4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28\" (UID: \"30a58c66-a2a9-4b16-bbad-b6927c94afb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" Jun 11 16:20:13.836299 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.836168 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30a58c66-a2a9-4b16-bbad-b6927c94afb4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28\" (UID: \"30a58c66-a2a9-4b16-bbad-b6927c94afb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" Jun 11 16:20:13.836555 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.836536 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30a58c66-a2a9-4b16-bbad-b6927c94afb4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28\" (UID: \"30a58c66-a2a9-4b16-bbad-b6927c94afb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" Jun 11 16:20:13.836601 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.836550 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30a58c66-a2a9-4b16-bbad-b6927c94afb4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28\" (UID: \"30a58c66-a2a9-4b16-bbad-b6927c94afb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" Jun 11 16:20:13.845449 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.845423 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghbdn\" (UniqueName: \"kubernetes.io/projected/30a58c66-a2a9-4b16-bbad-b6927c94afb4-kube-api-access-ghbdn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28\" (UID: \"30a58c66-a2a9-4b16-bbad-b6927c94afb4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" Jun 11 16:20:13.956828 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:13.956798 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" Jun 11 16:20:14.015965 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.015936 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j"] Jun 11 16:20:14.020347 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.020326 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" Jun 11 16:20:14.027418 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.027335 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j"] Jun 11 16:20:14.107121 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.107081 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28"] Jun 11 16:20:14.109769 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:20:14.109735 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30a58c66_a2a9_4b16_bbad_b6927c94afb4.slice/crio-b0e9615d7d5e6d6caf38d898cd61ecdbb8099a5a4707197c7b054b02b4ecbac8 WatchSource:0}: Error finding container b0e9615d7d5e6d6caf38d898cd61ecdbb8099a5a4707197c7b054b02b4ecbac8: Status 404 returned error can't find the container with id b0e9615d7d5e6d6caf38d898cd61ecdbb8099a5a4707197c7b054b02b4ecbac8 Jun 11 16:20:14.138351 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.138323 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8007009-308f-41bb-8554-5e34eb0ebd24-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j\" (UID: \"a8007009-308f-41bb-8554-5e34eb0ebd24\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" Jun 11 16:20:14.138463 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.138374 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8007009-308f-41bb-8554-5e34eb0ebd24-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j\" (UID: \"a8007009-308f-41bb-8554-5e34eb0ebd24\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" Jun 11 16:20:14.138463 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.138411 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgmfj\" (UniqueName: \"kubernetes.io/projected/a8007009-308f-41bb-8554-5e34eb0ebd24-kube-api-access-wgmfj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j\" (UID: \"a8007009-308f-41bb-8554-5e34eb0ebd24\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" Jun 11 16:20:14.225703 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.225669 2573 generic.go:356] "Generic (PLEG): container finished" podID="7366b4af-f894-4c26-8491-e6772f6a723d" containerID="4ae8885c069ee44b8db41927ae5204c3c8215a324e83f5a2c112f1d3c169494e" exitCode=0 Jun 11 16:20:14.225877 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.225751 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" event={"ID":"7366b4af-f894-4c26-8491-e6772f6a723d","Type":"ContainerDied","Data":"4ae8885c069ee44b8db41927ae5204c3c8215a324e83f5a2c112f1d3c169494e"} Jun 11 16:20:14.227391 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.227369 2573 generic.go:356] "Generic (PLEG): container finished" podID="071f0967-15ae-4799-a8c3-c31510360054" containerID="17b182dd3b2dfc151c667843ef28b71cd97576705f152ca84fd4ffb14611fc8a" exitCode=0 Jun 11 16:20:14.227466 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.227419 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" event={"ID":"071f0967-15ae-4799-a8c3-c31510360054","Type":"ContainerDied","Data":"17b182dd3b2dfc151c667843ef28b71cd97576705f152ca84fd4ffb14611fc8a"} Jun 11 16:20:14.227519 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.227463 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" event={"ID":"071f0967-15ae-4799-a8c3-c31510360054","Type":"ContainerStarted","Data":"6aa5ca67675efec5d26e2b923ff5c281efda1c877ce4b5e2da9e6c89ed45be6f"} Jun 11 16:20:14.229187 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.229159 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" event={"ID":"30a58c66-a2a9-4b16-bbad-b6927c94afb4","Type":"ContainerStarted","Data":"1f76cd1715fcb379f59a8b2344929ae4350a56436cc3632721f1faa85373f733"} Jun 11 16:20:14.229305 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.229209 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" event={"ID":"30a58c66-a2a9-4b16-bbad-b6927c94afb4","Type":"ContainerStarted","Data":"b0e9615d7d5e6d6caf38d898cd61ecdbb8099a5a4707197c7b054b02b4ecbac8"} Jun 11 16:20:14.239106 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.239079 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8007009-308f-41bb-8554-5e34eb0ebd24-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j\" (UID: \"a8007009-308f-41bb-8554-5e34eb0ebd24\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" Jun 11 16:20:14.239260 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.239136 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8007009-308f-41bb-8554-5e34eb0ebd24-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j\" (UID: \"a8007009-308f-41bb-8554-5e34eb0ebd24\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" Jun 11 16:20:14.239260 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.239179 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgmfj\" (UniqueName: \"kubernetes.io/projected/a8007009-308f-41bb-8554-5e34eb0ebd24-kube-api-access-wgmfj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j\" (UID: \"a8007009-308f-41bb-8554-5e34eb0ebd24\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" Jun 11 16:20:14.239455 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.239438 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8007009-308f-41bb-8554-5e34eb0ebd24-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j\" (UID: \"a8007009-308f-41bb-8554-5e34eb0ebd24\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" Jun 11 16:20:14.239522 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.239492 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8007009-308f-41bb-8554-5e34eb0ebd24-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j\" (UID: \"a8007009-308f-41bb-8554-5e34eb0ebd24\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" Jun 11 16:20:14.248809 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.248768 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgmfj\" (UniqueName: \"kubernetes.io/projected/a8007009-308f-41bb-8554-5e34eb0ebd24-kube-api-access-wgmfj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j\" (UID: \"a8007009-308f-41bb-8554-5e34eb0ebd24\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" Jun 11 16:20:14.336758 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.336737 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" Jun 11 16:20:14.496396 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:14.496366 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j"] Jun 11 16:20:14.496955 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:20:14.496910 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8007009_308f_41bb_8554_5e34eb0ebd24.slice/crio-d69794cc0e224825aeb33903a247127df73cbef8367f13f103ad2b8465a490f7 WatchSource:0}: Error finding container d69794cc0e224825aeb33903a247127df73cbef8367f13f103ad2b8465a490f7: Status 404 returned error can't find the container with id d69794cc0e224825aeb33903a247127df73cbef8367f13f103ad2b8465a490f7 Jun 11 16:20:15.235897 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:15.235857 2573 generic.go:356] "Generic (PLEG): container finished" podID="a8007009-308f-41bb-8554-5e34eb0ebd24" containerID="5f32882eeee1ee3fd588049e9e5a8d0f0807de22ca49cbcbe34d0401f60c1dd7" exitCode=0 Jun 11 16:20:15.236100 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:15.235899 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" event={"ID":"a8007009-308f-41bb-8554-5e34eb0ebd24","Type":"ContainerDied","Data":"5f32882eeee1ee3fd588049e9e5a8d0f0807de22ca49cbcbe34d0401f60c1dd7"} Jun 11 16:20:15.236100 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:15.235986 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" event={"ID":"a8007009-308f-41bb-8554-5e34eb0ebd24","Type":"ContainerStarted","Data":"d69794cc0e224825aeb33903a247127df73cbef8367f13f103ad2b8465a490f7"} Jun 11 16:20:15.237540 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:15.237510 2573 generic.go:356] "Generic (PLEG): container finished" podID="30a58c66-a2a9-4b16-bbad-b6927c94afb4" containerID="1f76cd1715fcb379f59a8b2344929ae4350a56436cc3632721f1faa85373f733" exitCode=0 Jun 11 16:20:15.237617 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:15.237576 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" event={"ID":"30a58c66-a2a9-4b16-bbad-b6927c94afb4","Type":"ContainerDied","Data":"1f76cd1715fcb379f59a8b2344929ae4350a56436cc3632721f1faa85373f733"} Jun 11 16:20:15.239725 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:15.239701 2573 generic.go:356] "Generic (PLEG): container finished" podID="7366b4af-f894-4c26-8491-e6772f6a723d" containerID="5f65a3bc3c50fb3ffabd10be38d7e587f882f38b3ba9009c4e5c88fb5af34d59" exitCode=0 Jun 11 16:20:15.239837 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:15.239793 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" event={"ID":"7366b4af-f894-4c26-8491-e6772f6a723d","Type":"ContainerDied","Data":"5f65a3bc3c50fb3ffabd10be38d7e587f882f38b3ba9009c4e5c88fb5af34d59"} Jun 11 16:20:15.241625 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:15.241604 2573 generic.go:356] "Generic (PLEG): container finished" podID="071f0967-15ae-4799-a8c3-c31510360054" containerID="5792e4dc98e351459a9756c089be49723b1ac7b82910c9f6c97c85dff2ff4f3b" exitCode=0 Jun 11 16:20:15.241714 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:15.241649 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" event={"ID":"071f0967-15ae-4799-a8c3-c31510360054","Type":"ContainerDied","Data":"5792e4dc98e351459a9756c089be49723b1ac7b82910c9f6c97c85dff2ff4f3b"} Jun 11 16:20:16.191178 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:20:16.191091 2573 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8007009_308f_41bb_8554_5e34eb0ebd24.slice/crio-e868ddf1a01def4f6a772c6b9864f2661f2e227a626a0182b6d4282bc2ff8404.scope\": RecentStats: unable to find data in memory cache]" Jun 11 16:20:16.249133 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:16.249098 2573 generic.go:356] "Generic (PLEG): container finished" podID="a8007009-308f-41bb-8554-5e34eb0ebd24" containerID="e868ddf1a01def4f6a772c6b9864f2661f2e227a626a0182b6d4282bc2ff8404" exitCode=0 Jun 11 16:20:16.249328 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:16.249190 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" event={"ID":"a8007009-308f-41bb-8554-5e34eb0ebd24","Type":"ContainerDied","Data":"e868ddf1a01def4f6a772c6b9864f2661f2e227a626a0182b6d4282bc2ff8404"} Jun 11 16:20:16.251124 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:16.251104 2573 generic.go:356] "Generic (PLEG): container finished" podID="30a58c66-a2a9-4b16-bbad-b6927c94afb4" containerID="29ae9302f260e788aa6628fe444516631a5cd4e62588e12193a89729d0709acd" exitCode=0 Jun 11 16:20:16.251214 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:16.251191 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" event={"ID":"30a58c66-a2a9-4b16-bbad-b6927c94afb4","Type":"ContainerDied","Data":"29ae9302f260e788aa6628fe444516631a5cd4e62588e12193a89729d0709acd"} Jun 11 16:20:16.253410 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:16.253391 2573 generic.go:356] "Generic (PLEG): container finished" podID="071f0967-15ae-4799-a8c3-c31510360054" containerID="24de8a2a823a48eff4f176761b8402a6d74b7a7056583b015b7a980f924b5d2a" exitCode=0 Jun 11 16:20:16.253489 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:16.253475 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" event={"ID":"071f0967-15ae-4799-a8c3-c31510360054","Type":"ContainerDied","Data":"24de8a2a823a48eff4f176761b8402a6d74b7a7056583b015b7a980f924b5d2a"} Jun 11 16:20:16.394419 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:16.394387 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" Jun 11 16:20:16.558519 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:16.558480 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kdvrw\" (UniqueName: \"kubernetes.io/projected/7366b4af-f894-4c26-8491-e6772f6a723d-kube-api-access-kdvrw\") pod \"7366b4af-f894-4c26-8491-e6772f6a723d\" (UID: \"7366b4af-f894-4c26-8491-e6772f6a723d\") " Jun 11 16:20:16.558692 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:16.558542 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7366b4af-f894-4c26-8491-e6772f6a723d-bundle\") pod \"7366b4af-f894-4c26-8491-e6772f6a723d\" (UID: \"7366b4af-f894-4c26-8491-e6772f6a723d\") " Jun 11 16:20:16.558692 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:16.558575 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7366b4af-f894-4c26-8491-e6772f6a723d-util\") pod \"7366b4af-f894-4c26-8491-e6772f6a723d\" (UID: \"7366b4af-f894-4c26-8491-e6772f6a723d\") " Jun 11 16:20:16.559159 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:16.559125 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7366b4af-f894-4c26-8491-e6772f6a723d-bundle" (OuterVolumeSpecName: "bundle") pod "7366b4af-f894-4c26-8491-e6772f6a723d" (UID: "7366b4af-f894-4c26-8491-e6772f6a723d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:20:16.561019 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:16.560995 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7366b4af-f894-4c26-8491-e6772f6a723d-kube-api-access-kdvrw" (OuterVolumeSpecName: "kube-api-access-kdvrw") pod "7366b4af-f894-4c26-8491-e6772f6a723d" (UID: "7366b4af-f894-4c26-8491-e6772f6a723d"). InnerVolumeSpecName "kube-api-access-kdvrw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:20:16.564057 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:16.564022 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7366b4af-f894-4c26-8491-e6772f6a723d-util" (OuterVolumeSpecName: "util") pod "7366b4af-f894-4c26-8491-e6772f6a723d" (UID: "7366b4af-f894-4c26-8491-e6772f6a723d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:20:16.659531 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:16.659489 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdvrw\" (UniqueName: \"kubernetes.io/projected/7366b4af-f894-4c26-8491-e6772f6a723d-kube-api-access-kdvrw\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:20:16.659531 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:16.659529 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7366b4af-f894-4c26-8491-e6772f6a723d-bundle\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:20:16.659783 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:16.659545 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7366b4af-f894-4c26-8491-e6772f6a723d-util\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:20:17.260535 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:17.260489 2573 generic.go:356] "Generic (PLEG): container finished" podID="a8007009-308f-41bb-8554-5e34eb0ebd24" containerID="846e1d4492b09361910c2874c5a4122ba3c0319f55dfcd5cb861c73c5e0a1fc5" exitCode=0 Jun 11 16:20:17.260970 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:17.260562 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" event={"ID":"a8007009-308f-41bb-8554-5e34eb0ebd24","Type":"ContainerDied","Data":"846e1d4492b09361910c2874c5a4122ba3c0319f55dfcd5cb861c73c5e0a1fc5"} Jun 11 16:20:17.262624 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:17.262601 2573 generic.go:356] "Generic (PLEG): container finished" podID="30a58c66-a2a9-4b16-bbad-b6927c94afb4" containerID="df4e78a4fec5f9164f7b71e31eaf7b52cf6662bb6b5c26b5159f871934694ac2" exitCode=0 Jun 11 16:20:17.262746 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:17.262682 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" event={"ID":"30a58c66-a2a9-4b16-bbad-b6927c94afb4","Type":"ContainerDied","Data":"df4e78a4fec5f9164f7b71e31eaf7b52cf6662bb6b5c26b5159f871934694ac2"} Jun 11 16:20:17.264523 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:17.264506 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" Jun 11 16:20:17.264610 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:17.264533 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc" event={"ID":"7366b4af-f894-4c26-8491-e6772f6a723d","Type":"ContainerDied","Data":"eecb7bb433fb29c7487d2414efdd3680590831d3c8d9695e9a7a06ddbd629100"} Jun 11 16:20:17.264610 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:17.264567 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eecb7bb433fb29c7487d2414efdd3680590831d3c8d9695e9a7a06ddbd629100" Jun 11 16:20:17.403147 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:17.403118 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" Jun 11 16:20:17.565870 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:17.565786 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/071f0967-15ae-4799-a8c3-c31510360054-util\") pod \"071f0967-15ae-4799-a8c3-c31510360054\" (UID: \"071f0967-15ae-4799-a8c3-c31510360054\") " Jun 11 16:20:17.565870 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:17.565844 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/071f0967-15ae-4799-a8c3-c31510360054-bundle\") pod \"071f0967-15ae-4799-a8c3-c31510360054\" (UID: \"071f0967-15ae-4799-a8c3-c31510360054\") " Jun 11 16:20:17.566089 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:17.565889 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxmjz\" (UniqueName: \"kubernetes.io/projected/071f0967-15ae-4799-a8c3-c31510360054-kube-api-access-lxmjz\") pod \"071f0967-15ae-4799-a8c3-c31510360054\" (UID: \"071f0967-15ae-4799-a8c3-c31510360054\") " Jun 11 16:20:17.566538 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:17.566506 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071f0967-15ae-4799-a8c3-c31510360054-bundle" (OuterVolumeSpecName: "bundle") pod "071f0967-15ae-4799-a8c3-c31510360054" (UID: "071f0967-15ae-4799-a8c3-c31510360054"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:20:17.568239 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:17.568218 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/071f0967-15ae-4799-a8c3-c31510360054-kube-api-access-lxmjz" (OuterVolumeSpecName: "kube-api-access-lxmjz") pod "071f0967-15ae-4799-a8c3-c31510360054" (UID: "071f0967-15ae-4799-a8c3-c31510360054"). InnerVolumeSpecName "kube-api-access-lxmjz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:20:17.571040 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:17.571006 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/071f0967-15ae-4799-a8c3-c31510360054-util" (OuterVolumeSpecName: "util") pod "071f0967-15ae-4799-a8c3-c31510360054" (UID: "071f0967-15ae-4799-a8c3-c31510360054"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:20:17.666906 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:17.666867 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/071f0967-15ae-4799-a8c3-c31510360054-util\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:20:17.666906 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:17.666896 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/071f0967-15ae-4799-a8c3-c31510360054-bundle\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:20:17.666906 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:17.666907 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxmjz\" (UniqueName: \"kubernetes.io/projected/071f0967-15ae-4799-a8c3-c31510360054-kube-api-access-lxmjz\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:20:18.272336 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.272307 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" Jun 11 16:20:18.272336 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.272319 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82" event={"ID":"071f0967-15ae-4799-a8c3-c31510360054","Type":"ContainerDied","Data":"6aa5ca67675efec5d26e2b923ff5c281efda1c877ce4b5e2da9e6c89ed45be6f"} Jun 11 16:20:18.272868 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.272362 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6aa5ca67675efec5d26e2b923ff5c281efda1c877ce4b5e2da9e6c89ed45be6f" Jun 11 16:20:18.433653 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.433633 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" Jun 11 16:20:18.435626 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.435609 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" Jun 11 16:20:18.575007 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.574931 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8007009-308f-41bb-8554-5e34eb0ebd24-util\") pod \"a8007009-308f-41bb-8554-5e34eb0ebd24\" (UID: \"a8007009-308f-41bb-8554-5e34eb0ebd24\") " Jun 11 16:20:18.575171 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.575028 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgmfj\" (UniqueName: \"kubernetes.io/projected/a8007009-308f-41bb-8554-5e34eb0ebd24-kube-api-access-wgmfj\") pod \"a8007009-308f-41bb-8554-5e34eb0ebd24\" (UID: \"a8007009-308f-41bb-8554-5e34eb0ebd24\") " Jun 11 16:20:18.575171 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.575057 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghbdn\" (UniqueName: \"kubernetes.io/projected/30a58c66-a2a9-4b16-bbad-b6927c94afb4-kube-api-access-ghbdn\") pod \"30a58c66-a2a9-4b16-bbad-b6927c94afb4\" (UID: \"30a58c66-a2a9-4b16-bbad-b6927c94afb4\") " Jun 11 16:20:18.575171 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.575086 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8007009-308f-41bb-8554-5e34eb0ebd24-bundle\") pod \"a8007009-308f-41bb-8554-5e34eb0ebd24\" (UID: \"a8007009-308f-41bb-8554-5e34eb0ebd24\") " Jun 11 16:20:18.575171 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.575146 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30a58c66-a2a9-4b16-bbad-b6927c94afb4-bundle\") pod \"30a58c66-a2a9-4b16-bbad-b6927c94afb4\" (UID: \"30a58c66-a2a9-4b16-bbad-b6927c94afb4\") " Jun 11 16:20:18.575171 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.575169 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30a58c66-a2a9-4b16-bbad-b6927c94afb4-util\") pod \"30a58c66-a2a9-4b16-bbad-b6927c94afb4\" (UID: \"30a58c66-a2a9-4b16-bbad-b6927c94afb4\") " Jun 11 16:20:18.575645 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.575613 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8007009-308f-41bb-8554-5e34eb0ebd24-bundle" (OuterVolumeSpecName: "bundle") pod "a8007009-308f-41bb-8554-5e34eb0ebd24" (UID: "a8007009-308f-41bb-8554-5e34eb0ebd24"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:20:18.575771 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.575674 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30a58c66-a2a9-4b16-bbad-b6927c94afb4-bundle" (OuterVolumeSpecName: "bundle") pod "30a58c66-a2a9-4b16-bbad-b6927c94afb4" (UID: "30a58c66-a2a9-4b16-bbad-b6927c94afb4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:20:18.577357 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.577332 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8007009-308f-41bb-8554-5e34eb0ebd24-kube-api-access-wgmfj" (OuterVolumeSpecName: "kube-api-access-wgmfj") pod "a8007009-308f-41bb-8554-5e34eb0ebd24" (UID: "a8007009-308f-41bb-8554-5e34eb0ebd24"). InnerVolumeSpecName "kube-api-access-wgmfj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:20:18.577484 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.577375 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30a58c66-a2a9-4b16-bbad-b6927c94afb4-kube-api-access-ghbdn" (OuterVolumeSpecName: "kube-api-access-ghbdn") pod "30a58c66-a2a9-4b16-bbad-b6927c94afb4" (UID: "30a58c66-a2a9-4b16-bbad-b6927c94afb4"). InnerVolumeSpecName "kube-api-access-ghbdn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:20:18.580890 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.580857 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8007009-308f-41bb-8554-5e34eb0ebd24-util" (OuterVolumeSpecName: "util") pod "a8007009-308f-41bb-8554-5e34eb0ebd24" (UID: "a8007009-308f-41bb-8554-5e34eb0ebd24"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:20:18.583307 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.583277 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30a58c66-a2a9-4b16-bbad-b6927c94afb4-util" (OuterVolumeSpecName: "util") pod "30a58c66-a2a9-4b16-bbad-b6927c94afb4" (UID: "30a58c66-a2a9-4b16-bbad-b6927c94afb4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:20:18.676742 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.676688 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/30a58c66-a2a9-4b16-bbad-b6927c94afb4-bundle\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:20:18.676742 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.676735 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/30a58c66-a2a9-4b16-bbad-b6927c94afb4-util\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:20:18.676742 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.676744 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a8007009-308f-41bb-8554-5e34eb0ebd24-util\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:20:18.676742 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.676753 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgmfj\" (UniqueName: \"kubernetes.io/projected/a8007009-308f-41bb-8554-5e34eb0ebd24-kube-api-access-wgmfj\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:20:18.677033 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.676764 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghbdn\" (UniqueName: \"kubernetes.io/projected/30a58c66-a2a9-4b16-bbad-b6927c94afb4-kube-api-access-ghbdn\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:20:18.677033 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:18.676772 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a8007009-308f-41bb-8554-5e34eb0ebd24-bundle\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:20:19.279962 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:19.279912 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" Jun 11 16:20:19.280359 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:19.279911 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j" event={"ID":"a8007009-308f-41bb-8554-5e34eb0ebd24","Type":"ContainerDied","Data":"d69794cc0e224825aeb33903a247127df73cbef8367f13f103ad2b8465a490f7"} Jun 11 16:20:19.280359 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:19.280056 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d69794cc0e224825aeb33903a247127df73cbef8367f13f103ad2b8465a490f7" Jun 11 16:20:19.282073 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:19.282045 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" Jun 11 16:20:19.282203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:19.282069 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28" event={"ID":"30a58c66-a2a9-4b16-bbad-b6927c94afb4","Type":"ContainerDied","Data":"b0e9615d7d5e6d6caf38d898cd61ecdbb8099a5a4707197c7b054b02b4ecbac8"} Jun 11 16:20:19.282203 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:19.282112 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b0e9615d7d5e6d6caf38d898cd61ecdbb8099a5a4707197c7b054b02b4ecbac8" Jun 11 16:20:36.148831 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.148741 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-q2msk"] Jun 11 16:20:36.149302 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149272 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7366b4af-f894-4c26-8491-e6772f6a723d" containerName="util" Jun 11 16:20:36.149302 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149292 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7366b4af-f894-4c26-8491-e6772f6a723d" containerName="util" Jun 11 16:20:36.149433 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149305 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="071f0967-15ae-4799-a8c3-c31510360054" containerName="extract" Jun 11 16:20:36.149433 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149315 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="071f0967-15ae-4799-a8c3-c31510360054" containerName="extract" Jun 11 16:20:36.149433 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149326 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30a58c66-a2a9-4b16-bbad-b6927c94afb4" containerName="pull" Jun 11 16:20:36.149433 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149334 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="30a58c66-a2a9-4b16-bbad-b6927c94afb4" containerName="pull" Jun 11 16:20:36.149433 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149351 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8007009-308f-41bb-8554-5e34eb0ebd24" containerName="pull" Jun 11 16:20:36.149433 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149359 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8007009-308f-41bb-8554-5e34eb0ebd24" containerName="pull" Jun 11 16:20:36.149433 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149383 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30a58c66-a2a9-4b16-bbad-b6927c94afb4" containerName="extract" Jun 11 16:20:36.149433 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149391 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="30a58c66-a2a9-4b16-bbad-b6927c94afb4" containerName="extract" Jun 11 16:20:36.149433 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149403 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7366b4af-f894-4c26-8491-e6772f6a723d" containerName="extract" Jun 11 16:20:36.149433 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149411 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7366b4af-f894-4c26-8491-e6772f6a723d" containerName="extract" Jun 11 16:20:36.149433 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149420 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8007009-308f-41bb-8554-5e34eb0ebd24" containerName="util" Jun 11 16:20:36.149433 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149428 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8007009-308f-41bb-8554-5e34eb0ebd24" containerName="util" Jun 11 16:20:36.149433 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149437 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8007009-308f-41bb-8554-5e34eb0ebd24" containerName="extract" Jun 11 16:20:36.150049 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149445 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8007009-308f-41bb-8554-5e34eb0ebd24" containerName="extract" Jun 11 16:20:36.150049 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149456 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="071f0967-15ae-4799-a8c3-c31510360054" containerName="pull" Jun 11 16:20:36.150049 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149464 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="071f0967-15ae-4799-a8c3-c31510360054" containerName="pull" Jun 11 16:20:36.150049 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149476 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="071f0967-15ae-4799-a8c3-c31510360054" containerName="util" Jun 11 16:20:36.150049 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149486 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="071f0967-15ae-4799-a8c3-c31510360054" containerName="util" Jun 11 16:20:36.150049 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149498 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7366b4af-f894-4c26-8491-e6772f6a723d" containerName="pull" Jun 11 16:20:36.150049 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149506 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7366b4af-f894-4c26-8491-e6772f6a723d" containerName="pull" Jun 11 16:20:36.150049 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149521 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30a58c66-a2a9-4b16-bbad-b6927c94afb4" containerName="util" Jun 11 16:20:36.150049 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149529 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="30a58c66-a2a9-4b16-bbad-b6927c94afb4" containerName="util" Jun 11 16:20:36.150049 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149613 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7366b4af-f894-4c26-8491-e6772f6a723d" containerName="extract" Jun 11 16:20:36.150049 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149624 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8007009-308f-41bb-8554-5e34eb0ebd24" containerName="extract" Jun 11 16:20:36.150049 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149637 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="071f0967-15ae-4799-a8c3-c31510360054" containerName="extract" Jun 11 16:20:36.150049 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.149649 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="30a58c66-a2a9-4b16-bbad-b6927c94afb4" containerName="extract" Jun 11 16:20:36.152893 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.152872 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q2msk" Jun 11 16:20:36.159050 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.159027 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-drql4\"" Jun 11 16:20:36.159787 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.159760 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 11 16:20:36.159894 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.159800 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 11 16:20:36.160076 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.160062 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 11 16:20:36.169425 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.169400 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-q2msk"] Jun 11 16:20:36.223136 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.223095 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj7bc\" (UniqueName: \"kubernetes.io/projected/c5faeb56-6b55-4eaf-be07-d0e90f97f2a4-kube-api-access-pj7bc\") pod \"dns-operator-controller-manager-864dd78c4c-q2msk\" (UID: \"c5faeb56-6b55-4eaf-be07-d0e90f97f2a4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q2msk" Jun 11 16:20:36.324244 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.324198 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj7bc\" (UniqueName: \"kubernetes.io/projected/c5faeb56-6b55-4eaf-be07-d0e90f97f2a4-kube-api-access-pj7bc\") pod \"dns-operator-controller-manager-864dd78c4c-q2msk\" (UID: \"c5faeb56-6b55-4eaf-be07-d0e90f97f2a4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q2msk" Jun 11 16:20:36.337198 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.337165 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj7bc\" (UniqueName: \"kubernetes.io/projected/c5faeb56-6b55-4eaf-be07-d0e90f97f2a4-kube-api-access-pj7bc\") pod \"dns-operator-controller-manager-864dd78c4c-q2msk\" (UID: \"c5faeb56-6b55-4eaf-be07-d0e90f97f2a4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q2msk" Jun 11 16:20:36.463626 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.463538 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q2msk" Jun 11 16:20:36.613983 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:36.613956 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-q2msk"] Jun 11 16:20:36.614742 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:20:36.614714 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5faeb56_6b55_4eaf_be07_d0e90f97f2a4.slice/crio-6da7c779f8dec8aab22c33c33285795a9dc778c373a6609abfcd684c6db53d93 WatchSource:0}: Error finding container 6da7c779f8dec8aab22c33c33285795a9dc778c373a6609abfcd684c6db53d93: Status 404 returned error can't find the container with id 6da7c779f8dec8aab22c33c33285795a9dc778c373a6609abfcd684c6db53d93 Jun 11 16:20:37.377453 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:37.377414 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q2msk" event={"ID":"c5faeb56-6b55-4eaf-be07-d0e90f97f2a4","Type":"ContainerStarted","Data":"6da7c779f8dec8aab22c33c33285795a9dc778c373a6609abfcd684c6db53d93"} Jun 11 16:20:39.185773 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:39.185732 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf"] Jun 11 16:20:39.189261 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:39.189240 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" Jun 11 16:20:39.193319 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:39.193290 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-2ppsx\"" Jun 11 16:20:39.209782 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:39.209744 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf"] Jun 11 16:20:39.250028 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:39.249988 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxrlx\" (UniqueName: \"kubernetes.io/projected/b6701413-07f0-469a-8fa3-48e56f4fed80-kube-api-access-qxrlx\") pod \"limitador-operator-controller-manager-859d7b78bd-q4sgf\" (UID: \"b6701413-07f0-469a-8fa3-48e56f4fed80\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" Jun 11 16:20:39.351411 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:39.351369 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxrlx\" (UniqueName: \"kubernetes.io/projected/b6701413-07f0-469a-8fa3-48e56f4fed80-kube-api-access-qxrlx\") pod \"limitador-operator-controller-manager-859d7b78bd-q4sgf\" (UID: \"b6701413-07f0-469a-8fa3-48e56f4fed80\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" Jun 11 16:20:39.364638 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:39.364611 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxrlx\" (UniqueName: \"kubernetes.io/projected/b6701413-07f0-469a-8fa3-48e56f4fed80-kube-api-access-qxrlx\") pod \"limitador-operator-controller-manager-859d7b78bd-q4sgf\" (UID: \"b6701413-07f0-469a-8fa3-48e56f4fed80\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" Jun 11 16:20:39.391416 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:39.391378 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q2msk" event={"ID":"c5faeb56-6b55-4eaf-be07-d0e90f97f2a4","Type":"ContainerStarted","Data":"51b2c0483d1569e15343765bc2972ad88717db69ba52e63058be12dc7e8878bc"} Jun 11 16:20:39.391618 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:39.391585 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q2msk" Jun 11 16:20:39.435480 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:39.435420 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q2msk" podStartSLOduration=1.250726391 podStartE2EDuration="3.435404534s" totalImagesPullingTime="2.184678143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:20:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:20:39.433736207 +0000 UTC m=+715.267848746" watchObservedRunningTime="2026-06-11 16:20:39.435404534 +0000 UTC m=+715.269517073" Jun 11 16:20:39.499913 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:39.499818 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" Jun 11 16:20:39.653211 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:39.653060 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf"] Jun 11 16:20:39.655998 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:20:39.655969 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6701413_07f0_469a_8fa3_48e56f4fed80.slice/crio-a32865314334ab6e4d4674ff1582272f093fcd59a78dc0e6f19da65d1cf76252 WatchSource:0}: Error finding container a32865314334ab6e4d4674ff1582272f093fcd59a78dc0e6f19da65d1cf76252: Status 404 returned error can't find the container with id a32865314334ab6e4d4674ff1582272f093fcd59a78dc0e6f19da65d1cf76252 Jun 11 16:20:40.398939 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:40.398890 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" event={"ID":"b6701413-07f0-469a-8fa3-48e56f4fed80","Type":"ContainerStarted","Data":"a32865314334ab6e4d4674ff1582272f093fcd59a78dc0e6f19da65d1cf76252"} Jun 11 16:20:42.418863 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:42.418817 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" event={"ID":"b6701413-07f0-469a-8fa3-48e56f4fed80","Type":"ContainerStarted","Data":"0a49000ba781c2c22954fd751e256337e9120584e20184940ebff19c42e3361b"} Jun 11 16:20:42.419293 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:42.418978 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" Jun 11 16:20:42.438270 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:42.438215 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" podStartSLOduration=1.320261454 podStartE2EDuration="3.438200212s" totalImagesPullingTime="2.117938758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:20:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:20:42.435405281 +0000 UTC m=+718.269517821" watchObservedRunningTime="2026-06-11 16:20:42.438200212 +0000 UTC m=+718.272312752" Jun 11 16:20:50.401524 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:50.401493 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-q2msk" Jun 11 16:20:53.427032 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:53.427001 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" Jun 11 16:20:55.317474 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.317432 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf"] Jun 11 16:20:55.317956 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.317679 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" podUID="b6701413-07f0-469a-8fa3-48e56f4fed80" containerName="manager" containerID="cri-o://0a49000ba781c2c22954fd751e256337e9120584e20184940ebff19c42e3361b" gracePeriod=2 Jun 11 16:20:55.331140 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.330916 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf"] Jun 11 16:20:55.351401 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:20:55.351358 2573 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"b6701413-07f0-469a-8fa3-48e56f4fed80\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-q4sgf\": pods \"limitador-operator-controller-manager-859d7b78bd-q4sgf\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" Jun 11 16:20:55.362116 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.362084 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdzzf"] Jun 11 16:20:55.362498 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.362481 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6701413-07f0-469a-8fa3-48e56f4fed80" containerName="manager" Jun 11 16:20:55.362569 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.362503 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6701413-07f0-469a-8fa3-48e56f4fed80" containerName="manager" Jun 11 16:20:55.362625 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.362613 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6701413-07f0-469a-8fa3-48e56f4fed80" containerName="manager" Jun 11 16:20:55.365870 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.365852 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdzzf" Jun 11 16:20:55.367869 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:20:55.367828 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q4sgf\" is forbidden: User \"system:node:ip-10-0-140-111.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-111.ec2.internal' and this object" podUID="b6701413-07f0-469a-8fa3-48e56f4fed80" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" Jun 11 16:20:55.384744 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.384713 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdzzf"] Jun 11 16:20:55.493568 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.493536 2573 generic.go:356] "Generic (PLEG): container finished" podID="b6701413-07f0-469a-8fa3-48e56f4fed80" containerID="0a49000ba781c2c22954fd751e256337e9120584e20184940ebff19c42e3361b" exitCode=0 Jun 11 16:20:55.496884 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.496859 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-955kh\" (UniqueName: \"kubernetes.io/projected/0ea44854-c7bb-494b-a38b-3837e18a0cc8-kube-api-access-955kh\") pod \"limitador-operator-controller-manager-859d7b78bd-fdzzf\" (UID: \"0ea44854-c7bb-494b-a38b-3837e18a0cc8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdzzf" Jun 11 16:20:55.574734 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.574677 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" Jun 11 16:20:55.576868 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:20:55.576839 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q4sgf\" is forbidden: User \"system:node:ip-10-0-140-111.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-111.ec2.internal' and this object" podUID="b6701413-07f0-469a-8fa3-48e56f4fed80" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" Jun 11 16:20:55.597935 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.597889 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-955kh\" (UniqueName: \"kubernetes.io/projected/0ea44854-c7bb-494b-a38b-3837e18a0cc8-kube-api-access-955kh\") pod \"limitador-operator-controller-manager-859d7b78bd-fdzzf\" (UID: \"0ea44854-c7bb-494b-a38b-3837e18a0cc8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdzzf" Jun 11 16:20:55.609473 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.609445 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-955kh\" (UniqueName: \"kubernetes.io/projected/0ea44854-c7bb-494b-a38b-3837e18a0cc8-kube-api-access-955kh\") pod \"limitador-operator-controller-manager-859d7b78bd-fdzzf\" (UID: \"0ea44854-c7bb-494b-a38b-3837e18a0cc8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdzzf" Jun 11 16:20:55.690595 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.690559 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdzzf" Jun 11 16:20:55.698759 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.698734 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxrlx\" (UniqueName: \"kubernetes.io/projected/b6701413-07f0-469a-8fa3-48e56f4fed80-kube-api-access-qxrlx\") pod \"b6701413-07f0-469a-8fa3-48e56f4fed80\" (UID: \"b6701413-07f0-469a-8fa3-48e56f4fed80\") " Jun 11 16:20:55.701323 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.701284 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6701413-07f0-469a-8fa3-48e56f4fed80-kube-api-access-qxrlx" (OuterVolumeSpecName: "kube-api-access-qxrlx") pod "b6701413-07f0-469a-8fa3-48e56f4fed80" (UID: "b6701413-07f0-469a-8fa3-48e56f4fed80"). InnerVolumeSpecName "kube-api-access-qxrlx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:20:55.799652 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.799535 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxrlx\" (UniqueName: \"kubernetes.io/projected/b6701413-07f0-469a-8fa3-48e56f4fed80-kube-api-access-qxrlx\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:20:55.846376 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:55.846277 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdzzf"] Jun 11 16:20:55.848827 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:20:55.848800 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ea44854_c7bb_494b_a38b_3837e18a0cc8.slice/crio-e68e2ae13f9dae89a90f533e2d3932261a2cb7973aa3dd2563f12a5539aa4800 WatchSource:0}: Error finding container e68e2ae13f9dae89a90f533e2d3932261a2cb7973aa3dd2563f12a5539aa4800: Status 404 returned error can't find the container with id e68e2ae13f9dae89a90f533e2d3932261a2cb7973aa3dd2563f12a5539aa4800 Jun 11 16:20:56.500373 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:56.500338 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" Jun 11 16:20:56.500373 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:56.500367 2573 scope.go:117] "RemoveContainer" containerID="0a49000ba781c2c22954fd751e256337e9120584e20184940ebff19c42e3361b" Jun 11 16:20:56.502186 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:56.502153 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdzzf" event={"ID":"0ea44854-c7bb-494b-a38b-3837e18a0cc8","Type":"ContainerStarted","Data":"9db3418ff5dedca8920df16010279c3605db4d059e017171d905bb4d813da80f"} Jun 11 16:20:56.502284 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:56.502214 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdzzf" event={"ID":"0ea44854-c7bb-494b-a38b-3837e18a0cc8","Type":"ContainerStarted","Data":"e68e2ae13f9dae89a90f533e2d3932261a2cb7973aa3dd2563f12a5539aa4800"} Jun 11 16:20:56.502284 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:56.502246 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdzzf" Jun 11 16:20:56.503872 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:20:56.503841 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q4sgf\" is forbidden: User \"system:node:ip-10-0-140-111.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-111.ec2.internal' and this object" podUID="b6701413-07f0-469a-8fa3-48e56f4fed80" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" Jun 11 16:20:56.506024 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:20:56.505994 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q4sgf\" is forbidden: User \"system:node:ip-10-0-140-111.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-111.ec2.internal' and this object" podUID="b6701413-07f0-469a-8fa3-48e56f4fed80" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" Jun 11 16:20:56.536892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:56.536837 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdzzf" podStartSLOduration=1.5368170970000001 podStartE2EDuration="1.536817097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:20:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:20:56.534703352 +0000 UTC m=+732.368815904" watchObservedRunningTime="2026-06-11 16:20:56.536817097 +0000 UTC m=+732.370929636" Jun 11 16:20:56.538413 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:20:56.538388 2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q4sgf\" is forbidden: User \"system:node:ip-10-0-140-111.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-111.ec2.internal' and this object" podUID="b6701413-07f0-469a-8fa3-48e56f4fed80" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q4sgf" Jun 11 16:20:56.569034 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:20:56.568998 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6701413-07f0-469a-8fa3-48e56f4fed80" path="/var/lib/kubelet/pods/b6701413-07f0-469a-8fa3-48e56f4fed80/volumes" Jun 11 16:21:07.510655 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:07.510611 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-fdzzf" Jun 11 16:21:25.587519 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:25.587481 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qvkbx"] Jun 11 16:21:25.594979 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:25.594959 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" Jun 11 16:21:25.597601 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:25.597578 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 11 16:21:25.597715 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:25.597668 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tx77h\"" Jun 11 16:21:25.604774 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:25.604747 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qvkbx"] Jun 11 16:21:25.650721 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:25.650690 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d0b1e93-3e34-4514-88c4-0ac2ab52abee-config-file\") pod \"limitador-limitador-64c94fd64f-qvkbx\" (UID: \"4d0b1e93-3e34-4514-88c4-0ac2ab52abee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" Jun 11 16:21:25.650876 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:25.650778 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7whvw\" (UniqueName: \"kubernetes.io/projected/4d0b1e93-3e34-4514-88c4-0ac2ab52abee-kube-api-access-7whvw\") pod \"limitador-limitador-64c94fd64f-qvkbx\" (UID: \"4d0b1e93-3e34-4514-88c4-0ac2ab52abee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" Jun 11 16:21:25.672646 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:25.672613 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qvkbx"] Jun 11 16:21:25.751276 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:25.751238 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d0b1e93-3e34-4514-88c4-0ac2ab52abee-config-file\") pod \"limitador-limitador-64c94fd64f-qvkbx\" (UID: \"4d0b1e93-3e34-4514-88c4-0ac2ab52abee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" Jun 11 16:21:25.751456 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:25.751304 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7whvw\" (UniqueName: \"kubernetes.io/projected/4d0b1e93-3e34-4514-88c4-0ac2ab52abee-kube-api-access-7whvw\") pod \"limitador-limitador-64c94fd64f-qvkbx\" (UID: \"4d0b1e93-3e34-4514-88c4-0ac2ab52abee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" Jun 11 16:21:25.751868 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:25.751848 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d0b1e93-3e34-4514-88c4-0ac2ab52abee-config-file\") pod \"limitador-limitador-64c94fd64f-qvkbx\" (UID: \"4d0b1e93-3e34-4514-88c4-0ac2ab52abee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" Jun 11 16:21:25.759997 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:25.759973 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7whvw\" (UniqueName: \"kubernetes.io/projected/4d0b1e93-3e34-4514-88c4-0ac2ab52abee-kube-api-access-7whvw\") pod \"limitador-limitador-64c94fd64f-qvkbx\" (UID: \"4d0b1e93-3e34-4514-88c4-0ac2ab52abee\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" Jun 11 16:21:25.906970 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:25.906916 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" Jun 11 16:21:26.051666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:26.051635 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qvkbx"] Jun 11 16:21:26.051983 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:21:26.051959 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d0b1e93_3e34_4514_88c4_0ac2ab52abee.slice/crio-3fa7cda840a6332ed77130473c35a87785dda2f148ce2af735612c9bf3558cc2 WatchSource:0}: Error finding container 3fa7cda840a6332ed77130473c35a87785dda2f148ce2af735612c9bf3558cc2: Status 404 returned error can't find the container with id 3fa7cda840a6332ed77130473c35a87785dda2f148ce2af735612c9bf3558cc2 Jun 11 16:21:26.672118 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:26.672081 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" event={"ID":"4d0b1e93-3e34-4514-88c4-0ac2ab52abee","Type":"ContainerStarted","Data":"3fa7cda840a6332ed77130473c35a87785dda2f148ce2af735612c9bf3558cc2"} Jun 11 16:21:29.691483 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:29.691447 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" event={"ID":"4d0b1e93-3e34-4514-88c4-0ac2ab52abee","Type":"ContainerStarted","Data":"d3aa94efa8dd96986467d0ee09ac03faad557d5dd6571d596971c5cff25f9a5d"} Jun 11 16:21:29.691855 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:29.691579 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" Jun 11 16:21:40.093193 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.093133 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" podStartSLOduration=12.25046171 podStartE2EDuration="15.093114334s" totalImagesPullingTime="2.842652624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:21:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:21:29.71512178 +0000 UTC m=+765.549234320" watchObservedRunningTime="2026-06-11 16:21:40.093114334 +0000 UTC m=+775.927226873" Jun 11 16:21:40.094995 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.094975 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qvkbx"] Jun 11 16:21:40.095167 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.095148 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" podUID="4d0b1e93-3e34-4514-88c4-0ac2ab52abee" containerName="limitador" containerID="cri-o://d3aa94efa8dd96986467d0ee09ac03faad557d5dd6571d596971c5cff25f9a5d" gracePeriod=30 Jun 11 16:21:40.095957 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.095842 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" Jun 11 16:21:40.654366 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.654303 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" Jun 11 16:21:40.671356 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.671327 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7whvw\" (UniqueName: \"kubernetes.io/projected/4d0b1e93-3e34-4514-88c4-0ac2ab52abee-kube-api-access-7whvw\") pod \"4d0b1e93-3e34-4514-88c4-0ac2ab52abee\" (UID: \"4d0b1e93-3e34-4514-88c4-0ac2ab52abee\") " Jun 11 16:21:40.671578 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.671369 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d0b1e93-3e34-4514-88c4-0ac2ab52abee-config-file\") pod \"4d0b1e93-3e34-4514-88c4-0ac2ab52abee\" (UID: \"4d0b1e93-3e34-4514-88c4-0ac2ab52abee\") " Jun 11 16:21:40.672453 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.671974 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d0b1e93-3e34-4514-88c4-0ac2ab52abee-config-file" (OuterVolumeSpecName: "config-file") pod "4d0b1e93-3e34-4514-88c4-0ac2ab52abee" (UID: "4d0b1e93-3e34-4514-88c4-0ac2ab52abee"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 11 16:21:40.674073 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.674043 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d0b1e93-3e34-4514-88c4-0ac2ab52abee-kube-api-access-7whvw" (OuterVolumeSpecName: "kube-api-access-7whvw") pod "4d0b1e93-3e34-4514-88c4-0ac2ab52abee" (UID: "4d0b1e93-3e34-4514-88c4-0ac2ab52abee"). InnerVolumeSpecName "kube-api-access-7whvw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:21:40.761017 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.760983 2573 generic.go:356] "Generic (PLEG): container finished" podID="4d0b1e93-3e34-4514-88c4-0ac2ab52abee" containerID="d3aa94efa8dd96986467d0ee09ac03faad557d5dd6571d596971c5cff25f9a5d" exitCode=0 Jun 11 16:21:40.761192 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.761032 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" Jun 11 16:21:40.761192 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.761069 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" event={"ID":"4d0b1e93-3e34-4514-88c4-0ac2ab52abee","Type":"ContainerDied","Data":"d3aa94efa8dd96986467d0ee09ac03faad557d5dd6571d596971c5cff25f9a5d"} Jun 11 16:21:40.761192 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.761129 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-qvkbx" event={"ID":"4d0b1e93-3e34-4514-88c4-0ac2ab52abee","Type":"ContainerDied","Data":"3fa7cda840a6332ed77130473c35a87785dda2f148ce2af735612c9bf3558cc2"} Jun 11 16:21:40.761192 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.761159 2573 scope.go:117] "RemoveContainer" containerID="d3aa94efa8dd96986467d0ee09ac03faad557d5dd6571d596971c5cff25f9a5d" Jun 11 16:21:40.770678 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.770659 2573 scope.go:117] "RemoveContainer" containerID="d3aa94efa8dd96986467d0ee09ac03faad557d5dd6571d596971c5cff25f9a5d" Jun 11 16:21:40.770954 ip-10-0-140-111 kubenswrapper[2573]: E0611 16:21:40.770932 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3aa94efa8dd96986467d0ee09ac03faad557d5dd6571d596971c5cff25f9a5d\": container with ID starting with d3aa94efa8dd96986467d0ee09ac03faad557d5dd6571d596971c5cff25f9a5d not found: ID does not exist" containerID="d3aa94efa8dd96986467d0ee09ac03faad557d5dd6571d596971c5cff25f9a5d" Jun 11 16:21:40.771026 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.770960 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3aa94efa8dd96986467d0ee09ac03faad557d5dd6571d596971c5cff25f9a5d"} err="failed to get container status \"d3aa94efa8dd96986467d0ee09ac03faad557d5dd6571d596971c5cff25f9a5d\": rpc error: code = NotFound desc = could not find container \"d3aa94efa8dd96986467d0ee09ac03faad557d5dd6571d596971c5cff25f9a5d\": container with ID starting with d3aa94efa8dd96986467d0ee09ac03faad557d5dd6571d596971c5cff25f9a5d not found: ID does not exist" Jun 11 16:21:40.772284 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.772263 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7whvw\" (UniqueName: \"kubernetes.io/projected/4d0b1e93-3e34-4514-88c4-0ac2ab52abee-kube-api-access-7whvw\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:21:40.772346 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.772286 2573 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4d0b1e93-3e34-4514-88c4-0ac2ab52abee-config-file\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:21:40.783659 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.783624 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qvkbx"] Jun 11 16:21:40.787987 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:40.787959 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-qvkbx"] Jun 11 16:21:41.617822 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:41.617787 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-5z6fs"] Jun 11 16:21:41.618535 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:41.618510 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d0b1e93-3e34-4514-88c4-0ac2ab52abee" containerName="limitador" Jun 11 16:21:41.618535 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:41.618537 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d0b1e93-3e34-4514-88c4-0ac2ab52abee" containerName="limitador" Jun 11 16:21:41.618684 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:41.618661 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d0b1e93-3e34-4514-88c4-0ac2ab52abee" containerName="limitador" Jun 11 16:21:41.622959 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:41.622936 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-5z6fs" Jun 11 16:21:41.625310 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:41.625292 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jun 11 16:21:41.631710 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:41.631688 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-5z6fs"] Jun 11 16:21:41.680721 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:41.680687 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pczn\" (UniqueName: \"kubernetes.io/projected/95c045b8-2ba0-4891-bd1b-bcbf849b1686-kube-api-access-7pczn\") pod \"postgres-69fb4594cf-5z6fs\" (UID: \"95c045b8-2ba0-4891-bd1b-bcbf849b1686\") " pod="opendatahub/postgres-69fb4594cf-5z6fs" Jun 11 16:21:41.680893 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:41.680747 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/95c045b8-2ba0-4891-bd1b-bcbf849b1686-data\") pod \"postgres-69fb4594cf-5z6fs\" (UID: \"95c045b8-2ba0-4891-bd1b-bcbf849b1686\") " pod="opendatahub/postgres-69fb4594cf-5z6fs" Jun 11 16:21:41.781611 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:41.781573 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pczn\" (UniqueName: \"kubernetes.io/projected/95c045b8-2ba0-4891-bd1b-bcbf849b1686-kube-api-access-7pczn\") pod \"postgres-69fb4594cf-5z6fs\" (UID: \"95c045b8-2ba0-4891-bd1b-bcbf849b1686\") " pod="opendatahub/postgres-69fb4594cf-5z6fs" Jun 11 16:21:41.781764 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:41.781632 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/95c045b8-2ba0-4891-bd1b-bcbf849b1686-data\") pod \"postgres-69fb4594cf-5z6fs\" (UID: \"95c045b8-2ba0-4891-bd1b-bcbf849b1686\") " pod="opendatahub/postgres-69fb4594cf-5z6fs" Jun 11 16:21:41.782092 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:41.782065 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/95c045b8-2ba0-4891-bd1b-bcbf849b1686-data\") pod \"postgres-69fb4594cf-5z6fs\" (UID: \"95c045b8-2ba0-4891-bd1b-bcbf849b1686\") " pod="opendatahub/postgres-69fb4594cf-5z6fs" Jun 11 16:21:41.790971 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:41.790945 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pczn\" (UniqueName: \"kubernetes.io/projected/95c045b8-2ba0-4891-bd1b-bcbf849b1686-kube-api-access-7pczn\") pod \"postgres-69fb4594cf-5z6fs\" (UID: \"95c045b8-2ba0-4891-bd1b-bcbf849b1686\") " pod="opendatahub/postgres-69fb4594cf-5z6fs" Jun 11 16:21:41.933897 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:41.933860 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-5z6fs" Jun 11 16:21:42.071985 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:42.071954 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-5z6fs"] Jun 11 16:21:42.073879 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:21:42.073847 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95c045b8_2ba0_4891_bd1b_bcbf849b1686.slice/crio-906521c0d81a0469370169e56f3626fa85b983ab38e6e05421420cf62fc62b20 WatchSource:0}: Error finding container 906521c0d81a0469370169e56f3626fa85b983ab38e6e05421420cf62fc62b20: Status 404 returned error can't find the container with id 906521c0d81a0469370169e56f3626fa85b983ab38e6e05421420cf62fc62b20 Jun 11 16:21:42.568550 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:42.568514 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d0b1e93-3e34-4514-88c4-0ac2ab52abee" path="/var/lib/kubelet/pods/4d0b1e93-3e34-4514-88c4-0ac2ab52abee/volumes" Jun 11 16:21:42.776049 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:42.776007 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-5z6fs" event={"ID":"95c045b8-2ba0-4891-bd1b-bcbf849b1686","Type":"ContainerStarted","Data":"906521c0d81a0469370169e56f3626fa85b983ab38e6e05421420cf62fc62b20"} Jun 11 16:21:44.789997 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:44.789959 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-5z6fs" event={"ID":"95c045b8-2ba0-4891-bd1b-bcbf849b1686","Type":"ContainerStarted","Data":"ff40a4d39bbdd12a8bdc16c2d70f0b297dbc007084eae6b849d140a47ac00eb4"} Jun 11 16:21:44.790382 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:44.790086 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-5z6fs" Jun 11 16:21:44.814835 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:44.814784 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-5z6fs" podStartSLOduration=1.735329894 podStartE2EDuration="3.814766361s" totalImagesPullingTime="2.079436467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:21:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:21:44.812945671 +0000 UTC m=+780.647058211" watchObservedRunningTime="2026-06-11 16:21:44.814766361 +0000 UTC m=+780.648878899" Jun 11 16:21:50.829169 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:50.829138 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-5z6fs" Jun 11 16:21:53.465973 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:53.465940 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz"] Jun 11 16:21:53.472611 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:53.472593 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" Jun 11 16:21:53.476489 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:53.476470 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 11 16:21:53.476910 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:53.476889 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 11 16:21:53.478768 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:53.478748 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qh6np\"" Jun 11 16:21:53.487283 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:53.487261 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz"] Jun 11 16:21:53.579882 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:53.579846 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c2d9cd0-d53f-4707-b172-b80a0485e732-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz\" (UID: \"2c2d9cd0-d53f-4707-b172-b80a0485e732\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" Jun 11 16:21:53.580142 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:53.579896 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c2d9cd0-d53f-4707-b172-b80a0485e732-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz\" (UID: \"2c2d9cd0-d53f-4707-b172-b80a0485e732\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" Jun 11 16:21:53.580142 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:53.579958 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbpzt\" (UniqueName: \"kubernetes.io/projected/2c2d9cd0-d53f-4707-b172-b80a0485e732-kube-api-access-dbpzt\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz\" (UID: \"2c2d9cd0-d53f-4707-b172-b80a0485e732\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" Jun 11 16:21:53.681240 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:53.681198 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c2d9cd0-d53f-4707-b172-b80a0485e732-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz\" (UID: \"2c2d9cd0-d53f-4707-b172-b80a0485e732\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" Jun 11 16:21:53.681455 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:53.681284 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c2d9cd0-d53f-4707-b172-b80a0485e732-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz\" (UID: \"2c2d9cd0-d53f-4707-b172-b80a0485e732\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" Jun 11 16:21:53.681455 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:53.681316 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbpzt\" (UniqueName: \"kubernetes.io/projected/2c2d9cd0-d53f-4707-b172-b80a0485e732-kube-api-access-dbpzt\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz\" (UID: \"2c2d9cd0-d53f-4707-b172-b80a0485e732\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" Jun 11 16:21:53.681688 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:53.681665 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c2d9cd0-d53f-4707-b172-b80a0485e732-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz\" (UID: \"2c2d9cd0-d53f-4707-b172-b80a0485e732\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" Jun 11 16:21:53.681755 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:53.681704 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c2d9cd0-d53f-4707-b172-b80a0485e732-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz\" (UID: \"2c2d9cd0-d53f-4707-b172-b80a0485e732\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" Jun 11 16:21:53.692463 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:53.692431 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbpzt\" (UniqueName: \"kubernetes.io/projected/2c2d9cd0-d53f-4707-b172-b80a0485e732-kube-api-access-dbpzt\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz\" (UID: \"2c2d9cd0-d53f-4707-b172-b80a0485e732\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" Jun 11 16:21:53.782575 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:53.782482 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" Jun 11 16:21:53.933146 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:53.933120 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz"] Jun 11 16:21:53.935021 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:21:53.934989 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c2d9cd0_d53f_4707_b172_b80a0485e732.slice/crio-3ef4d66775c0f21a4b7be9638dd0502e1a9ee5edc464769ec7191f0272b61f50 WatchSource:0}: Error finding container 3ef4d66775c0f21a4b7be9638dd0502e1a9ee5edc464769ec7191f0272b61f50: Status 404 returned error can't find the container with id 3ef4d66775c0f21a4b7be9638dd0502e1a9ee5edc464769ec7191f0272b61f50 Jun 11 16:21:54.856049 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:54.856016 2573 generic.go:356] "Generic (PLEG): container finished" podID="2c2d9cd0-d53f-4707-b172-b80a0485e732" containerID="a9cb42d16a6fa65bbeb3813a55a8921ff304d8552692aa9b73b4c0b4400f418f" exitCode=0 Jun 11 16:21:54.856491 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:54.856101 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" event={"ID":"2c2d9cd0-d53f-4707-b172-b80a0485e732","Type":"ContainerDied","Data":"a9cb42d16a6fa65bbeb3813a55a8921ff304d8552692aa9b73b4c0b4400f418f"} Jun 11 16:21:54.856491 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:54.856163 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" event={"ID":"2c2d9cd0-d53f-4707-b172-b80a0485e732","Type":"ContainerStarted","Data":"3ef4d66775c0f21a4b7be9638dd0502e1a9ee5edc464769ec7191f0272b61f50"} Jun 11 16:21:55.864301 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:55.864267 2573 generic.go:356] "Generic (PLEG): container finished" podID="2c2d9cd0-d53f-4707-b172-b80a0485e732" containerID="b2517ef3b478e8d34dfa9853c709a3ea3df1d7e6de709601dbfe8bd5250bfba7" exitCode=0 Jun 11 16:21:55.864701 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:55.864346 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" event={"ID":"2c2d9cd0-d53f-4707-b172-b80a0485e732","Type":"ContainerDied","Data":"b2517ef3b478e8d34dfa9853c709a3ea3df1d7e6de709601dbfe8bd5250bfba7"} Jun 11 16:21:56.872795 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:56.872765 2573 generic.go:356] "Generic (PLEG): container finished" podID="2c2d9cd0-d53f-4707-b172-b80a0485e732" containerID="93ed55d2872375c86a8c7c405258f3e82d99e8219ecc16845a30cdbf4616b3ad" exitCode=0 Jun 11 16:21:56.873315 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:56.872852 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" event={"ID":"2c2d9cd0-d53f-4707-b172-b80a0485e732","Type":"ContainerDied","Data":"93ed55d2872375c86a8c7c405258f3e82d99e8219ecc16845a30cdbf4616b3ad"} Jun 11 16:21:58.024694 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:58.024668 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" Jun 11 16:21:58.118163 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:58.118130 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c2d9cd0-d53f-4707-b172-b80a0485e732-bundle\") pod \"2c2d9cd0-d53f-4707-b172-b80a0485e732\" (UID: \"2c2d9cd0-d53f-4707-b172-b80a0485e732\") " Jun 11 16:21:58.118351 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:58.118211 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c2d9cd0-d53f-4707-b172-b80a0485e732-util\") pod \"2c2d9cd0-d53f-4707-b172-b80a0485e732\" (UID: \"2c2d9cd0-d53f-4707-b172-b80a0485e732\") " Jun 11 16:21:58.118351 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:58.118266 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbpzt\" (UniqueName: \"kubernetes.io/projected/2c2d9cd0-d53f-4707-b172-b80a0485e732-kube-api-access-dbpzt\") pod \"2c2d9cd0-d53f-4707-b172-b80a0485e732\" (UID: \"2c2d9cd0-d53f-4707-b172-b80a0485e732\") " Jun 11 16:21:58.118666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:58.118636 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c2d9cd0-d53f-4707-b172-b80a0485e732-bundle" (OuterVolumeSpecName: "bundle") pod "2c2d9cd0-d53f-4707-b172-b80a0485e732" (UID: "2c2d9cd0-d53f-4707-b172-b80a0485e732"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:21:58.120575 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:58.120551 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c2d9cd0-d53f-4707-b172-b80a0485e732-kube-api-access-dbpzt" (OuterVolumeSpecName: "kube-api-access-dbpzt") pod "2c2d9cd0-d53f-4707-b172-b80a0485e732" (UID: "2c2d9cd0-d53f-4707-b172-b80a0485e732"). InnerVolumeSpecName "kube-api-access-dbpzt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:21:58.126030 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:58.125999 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c2d9cd0-d53f-4707-b172-b80a0485e732-util" (OuterVolumeSpecName: "util") pod "2c2d9cd0-d53f-4707-b172-b80a0485e732" (UID: "2c2d9cd0-d53f-4707-b172-b80a0485e732"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 11 16:21:58.219050 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:58.219012 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbpzt\" (UniqueName: \"kubernetes.io/projected/2c2d9cd0-d53f-4707-b172-b80a0485e732-kube-api-access-dbpzt\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:21:58.219050 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:58.219041 2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c2d9cd0-d53f-4707-b172-b80a0485e732-bundle\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:21:58.219050 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:58.219051 2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c2d9cd0-d53f-4707-b172-b80a0485e732-util\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:21:58.884796 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:58.884708 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" Jun 11 16:21:58.884796 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:58.884719 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748422fz" event={"ID":"2c2d9cd0-d53f-4707-b172-b80a0485e732","Type":"ContainerDied","Data":"3ef4d66775c0f21a4b7be9638dd0502e1a9ee5edc464769ec7191f0272b61f50"} Jun 11 16:21:58.884796 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:21:58.884770 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ef4d66775c0f21a4b7be9638dd0502e1a9ee5edc464769ec7191f0272b61f50" Jun 11 16:22:02.676904 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:02.676818 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-s5t95"] Jun 11 16:22:02.677330 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:02.677289 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c2d9cd0-d53f-4707-b172-b80a0485e732" containerName="extract" Jun 11 16:22:02.677330 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:02.677308 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c2d9cd0-d53f-4707-b172-b80a0485e732" containerName="extract" Jun 11 16:22:02.677330 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:02.677329 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c2d9cd0-d53f-4707-b172-b80a0485e732" containerName="util" Jun 11 16:22:02.677476 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:02.677338 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c2d9cd0-d53f-4707-b172-b80a0485e732" containerName="util" Jun 11 16:22:02.677476 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:02.677365 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c2d9cd0-d53f-4707-b172-b80a0485e732" containerName="pull" Jun 11 16:22:02.677476 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:02.677374 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c2d9cd0-d53f-4707-b172-b80a0485e732" containerName="pull" Jun 11 16:22:02.677476 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:02.677462 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c2d9cd0-d53f-4707-b172-b80a0485e732" containerName="extract" Jun 11 16:22:02.681148 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:02.681111 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-s5t95" Jun 11 16:22:02.684371 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:02.684348 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 11 16:22:02.685135 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:02.685113 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 11 16:22:02.685243 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:02.685113 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"keycloak-operator-dockercfg-b4xj5\"" Jun 11 16:22:02.711189 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:02.711110 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-s5t95"] Jun 11 16:22:02.760915 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:02.760878 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghtv9\" (UniqueName: \"kubernetes.io/projected/d2f28f75-65e0-408f-ac90-720eadb46091-kube-api-access-ghtv9\") pod \"keycloak-operator-d67dbbb48-s5t95\" (UID: \"d2f28f75-65e0-408f-ac90-720eadb46091\") " pod="keycloak-system/keycloak-operator-d67dbbb48-s5t95" Jun 11 16:22:02.861742 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:02.861700 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghtv9\" (UniqueName: \"kubernetes.io/projected/d2f28f75-65e0-408f-ac90-720eadb46091-kube-api-access-ghtv9\") pod \"keycloak-operator-d67dbbb48-s5t95\" (UID: \"d2f28f75-65e0-408f-ac90-720eadb46091\") " pod="keycloak-system/keycloak-operator-d67dbbb48-s5t95" Jun 11 16:22:02.879117 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:02.879078 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghtv9\" (UniqueName: \"kubernetes.io/projected/d2f28f75-65e0-408f-ac90-720eadb46091-kube-api-access-ghtv9\") pod \"keycloak-operator-d67dbbb48-s5t95\" (UID: \"d2f28f75-65e0-408f-ac90-720eadb46091\") " pod="keycloak-system/keycloak-operator-d67dbbb48-s5t95" Jun 11 16:22:02.992404 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:02.992309 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-s5t95" Jun 11 16:22:03.151558 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:03.151529 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-s5t95"] Jun 11 16:22:03.152812 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:22:03.152786 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2f28f75_65e0_408f_ac90_720eadb46091.slice/crio-ce6e22024ce7587ee412b62f96515835a199952357fc495a9d4a4ba431569168 WatchSource:0}: Error finding container ce6e22024ce7587ee412b62f96515835a199952357fc495a9d4a4ba431569168: Status 404 returned error can't find the container with id ce6e22024ce7587ee412b62f96515835a199952357fc495a9d4a4ba431569168 Jun 11 16:22:03.916029 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:03.915989 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-s5t95" event={"ID":"d2f28f75-65e0-408f-ac90-720eadb46091","Type":"ContainerStarted","Data":"ce6e22024ce7587ee412b62f96515835a199952357fc495a9d4a4ba431569168"} Jun 11 16:22:10.963236 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:10.963179 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-s5t95" event={"ID":"d2f28f75-65e0-408f-ac90-720eadb46091","Type":"ContainerStarted","Data":"3421c67da3e99264fe5031425778d924751a86506e963e523d3f9c51cfc94160"} Jun 11 16:22:10.989152 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:22:10.989095 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/keycloak-operator-d67dbbb48-s5t95" podStartSLOduration=2.054118535 podStartE2EDuration="8.989077768s" totalImagesPullingTime="6.934959233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:22:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:22:10.987044747 +0000 UTC m=+806.821157285" watchObservedRunningTime="2026-06-11 16:22:10.989077768 +0000 UTC m=+806.823190317" Jun 11 16:23:44.583647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:23:44.583618 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p7nvz_7acea643-2978-4893-a6a9-bf88a11b531b/ovn-acl-logging/0.log" Jun 11 16:23:44.586301 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:23:44.586282 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p7nvz_7acea643-2978-4893-a6a9-bf88a11b531b/ovn-acl-logging/0.log" Jun 11 16:26:51.135979 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.135949 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p"] Jun 11 16:26:51.139806 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.139785 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.142075 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.142053 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 11 16:26:51.142189 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.142074 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-26ghs\"" Jun 11 16:26:51.142647 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.142628 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 11 16:26:51.142730 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.142670 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 11 16:26:51.154991 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.154960 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p"] Jun 11 16:26:51.265045 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.265014 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0fafbdec-54e4-44d5-b7de-ea2e589dc926-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.265204 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.265077 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fafbdec-54e4-44d5-b7de-ea2e589dc926-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.265204 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.265117 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0fafbdec-54e4-44d5-b7de-ea2e589dc926-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.265204 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.265145 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg89r\" (UniqueName: \"kubernetes.io/projected/0fafbdec-54e4-44d5-b7de-ea2e589dc926-kube-api-access-xg89r\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.265204 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.265187 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0fafbdec-54e4-44d5-b7de-ea2e589dc926-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.265341 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.265236 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0fafbdec-54e4-44d5-b7de-ea2e589dc926-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.366074 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.366032 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0fafbdec-54e4-44d5-b7de-ea2e589dc926-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.366272 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.366112 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fafbdec-54e4-44d5-b7de-ea2e589dc926-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.366272 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.366144 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0fafbdec-54e4-44d5-b7de-ea2e589dc926-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.366272 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.366178 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg89r\" (UniqueName: \"kubernetes.io/projected/0fafbdec-54e4-44d5-b7de-ea2e589dc926-kube-api-access-xg89r\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.366425 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.366308 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0fafbdec-54e4-44d5-b7de-ea2e589dc926-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.366425 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.366355 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0fafbdec-54e4-44d5-b7de-ea2e589dc926-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.366513 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.366495 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0fafbdec-54e4-44d5-b7de-ea2e589dc926-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.366550 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.366538 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0fafbdec-54e4-44d5-b7de-ea2e589dc926-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.366704 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.366676 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0fafbdec-54e4-44d5-b7de-ea2e589dc926-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.368572 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.368548 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0fafbdec-54e4-44d5-b7de-ea2e589dc926-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.368812 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.368795 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0fafbdec-54e4-44d5-b7de-ea2e589dc926-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.376027 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.376000 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg89r\" (UniqueName: \"kubernetes.io/projected/0fafbdec-54e4-44d5-b7de-ea2e589dc926-kube-api-access-xg89r\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-nfw5p\" (UID: \"0fafbdec-54e4-44d5-b7de-ea2e589dc926\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.450116 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.450075 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" Jun 11 16:26:51.596664 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.596639 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p"] Jun 11 16:26:51.598679 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:26:51.598652 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fafbdec_54e4_44d5_b7de_ea2e589dc926.slice/crio-2ed0f1d94cf4f1981c0e4c1715137285571a6bed4f488097f99bde65a9394968 WatchSource:0}: Error finding container 2ed0f1d94cf4f1981c0e4c1715137285571a6bed4f488097f99bde65a9394968: Status 404 returned error can't find the container with id 2ed0f1d94cf4f1981c0e4c1715137285571a6bed4f488097f99bde65a9394968 Jun 11 16:26:51.600936 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:51.600907 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 11 16:26:52.582481 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:52.582443 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" event={"ID":"0fafbdec-54e4-44d5-b7de-ea2e589dc926","Type":"ContainerStarted","Data":"2ed0f1d94cf4f1981c0e4c1715137285571a6bed4f488097f99bde65a9394968"} Jun 11 16:26:56.609554 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:26:56.609513 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-nfw5p" event={"ID":"0fafbdec-54e4-44d5-b7de-ea2e589dc926","Type":"ContainerStarted","Data":"5c9304575ccfa420c1c4c0db5c5abf36a6a2b7cb62a5ca3ff6b18c6a9fec875d"} Jun 11 16:27:17.270205 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.270167 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22"] Jun 11 16:27:17.289445 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.289195 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.293944 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.292020 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 11 16:27:17.294162 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.294131 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22"] Jun 11 16:27:17.380894 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.380847 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6658471a-3a3a-4085-bdef-d9c19b7c4389-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.380894 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.380900 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6658471a-3a3a-4085-bdef-d9c19b7c4389-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.381146 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.380998 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g958z\" (UniqueName: \"kubernetes.io/projected/6658471a-3a3a-4085-bdef-d9c19b7c4389-kube-api-access-g958z\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.381146 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.381061 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6658471a-3a3a-4085-bdef-d9c19b7c4389-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.381146 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.381084 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6658471a-3a3a-4085-bdef-d9c19b7c4389-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.381247 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.381160 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6658471a-3a3a-4085-bdef-d9c19b7c4389-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.482597 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.482554 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6658471a-3a3a-4085-bdef-d9c19b7c4389-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.482597 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.482597 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6658471a-3a3a-4085-bdef-d9c19b7c4389-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.482829 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.482622 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6658471a-3a3a-4085-bdef-d9c19b7c4389-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.482829 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.482651 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g958z\" (UniqueName: \"kubernetes.io/projected/6658471a-3a3a-4085-bdef-d9c19b7c4389-kube-api-access-g958z\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.482829 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.482688 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6658471a-3a3a-4085-bdef-d9c19b7c4389-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.482829 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.482713 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6658471a-3a3a-4085-bdef-d9c19b7c4389-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.483142 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.483117 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6658471a-3a3a-4085-bdef-d9c19b7c4389-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.483211 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.483144 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6658471a-3a3a-4085-bdef-d9c19b7c4389-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.483272 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.483235 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6658471a-3a3a-4085-bdef-d9c19b7c4389-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.485233 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.485206 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6658471a-3a3a-4085-bdef-d9c19b7c4389-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.485375 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.485359 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6658471a-3a3a-4085-bdef-d9c19b7c4389-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.493912 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.493882 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g958z\" (UniqueName: \"kubernetes.io/projected/6658471a-3a3a-4085-bdef-d9c19b7c4389-kube-api-access-g958z\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-qgg22\" (UID: \"6658471a-3a3a-4085-bdef-d9c19b7c4389\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.602983 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.602861 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:27:17.758544 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:17.758508 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22"] Jun 11 16:27:17.760525 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:27:17.760490 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6658471a_3a3a_4085_bdef_d9c19b7c4389.slice/crio-ad54f6ee29824b891dd426b2cca1b72b4111b828ee9e2604c693b3d7946a1dad WatchSource:0}: Error finding container ad54f6ee29824b891dd426b2cca1b72b4111b828ee9e2604c693b3d7946a1dad: Status 404 returned error can't find the container with id ad54f6ee29824b891dd426b2cca1b72b4111b828ee9e2604c693b3d7946a1dad Jun 11 16:27:18.737551 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:18.737501 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" event={"ID":"6658471a-3a3a-4085-bdef-d9c19b7c4389","Type":"ContainerStarted","Data":"e1e3b7edfaeb890248e9d644f7b703870b1bd3fdab80b32f92d490b81717ca82"} Jun 11 16:27:18.738005 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:27:18.737584 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" event={"ID":"6658471a-3a3a-4085-bdef-d9c19b7c4389","Type":"ContainerStarted","Data":"ad54f6ee29824b891dd426b2cca1b72b4111b828ee9e2604c693b3d7946a1dad"} Jun 11 16:28:11.047972 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:28:11.047905 2573 generic.go:356] "Generic (PLEG): container finished" podID="6658471a-3a3a-4085-bdef-d9c19b7c4389" containerID="e1e3b7edfaeb890248e9d644f7b703870b1bd3fdab80b32f92d490b81717ca82" exitCode=0 Jun 11 16:28:11.048388 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:28:11.047980 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" event={"ID":"6658471a-3a3a-4085-bdef-d9c19b7c4389","Type":"ContainerDied","Data":"e1e3b7edfaeb890248e9d644f7b703870b1bd3fdab80b32f92d490b81717ca82"} Jun 11 16:28:13.063055 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:28:13.063020 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" event={"ID":"6658471a-3a3a-4085-bdef-d9c19b7c4389","Type":"ContainerStarted","Data":"20ff085592c3526578d7fd0462cde3269c8eef87b3404efa2b097e0947d79792"} Jun 11 16:28:13.085576 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:28:13.085515 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" podStartSLOduration=2.242336205 podStartE2EDuration="56.085498185s" totalImagesPullingTime="1.157812124s" totalInitContainerRuntime="52.685349856s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:27:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:28:13.083608623 +0000 UTC m=+1168.917721161" watchObservedRunningTime="2026-06-11 16:28:13.085498185 +0000 UTC m=+1168.919610723" Jun 11 16:28:17.603416 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:28:17.603376 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:28:17.603879 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:28:17.603448 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:28:17.615941 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:28:17.615902 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:28:18.106255 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:28:18.106227 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-qgg22" Jun 11 16:28:44.622219 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:28:44.622187 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p7nvz_7acea643-2978-4893-a6a9-bf88a11b531b/ovn-acl-logging/0.log" Jun 11 16:28:44.628326 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:28:44.628303 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p7nvz_7acea643-2978-4893-a6a9-bf88a11b531b/ovn-acl-logging/0.log" Jun 11 16:30:00.157784 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:00.157748 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29686590-6gdcd"] Jun 11 16:30:00.161276 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:00.161259 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29686590-6gdcd" Jun 11 16:30:00.164418 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:00.164385 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-cxvqj\"" Jun 11 16:30:00.180851 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:00.180822 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29686590-6gdcd"] Jun 11 16:30:00.203798 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:00.203760 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2scq\" (UniqueName: \"kubernetes.io/projected/0aa1239e-321d-4e0e-8183-6fa1361c0fae-kube-api-access-m2scq\") pod \"maas-api-key-cleanup-29686590-6gdcd\" (UID: \"0aa1239e-321d-4e0e-8183-6fa1361c0fae\") " pod="opendatahub/maas-api-key-cleanup-29686590-6gdcd" Jun 11 16:30:00.304559 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:00.304520 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2scq\" (UniqueName: \"kubernetes.io/projected/0aa1239e-321d-4e0e-8183-6fa1361c0fae-kube-api-access-m2scq\") pod \"maas-api-key-cleanup-29686590-6gdcd\" (UID: \"0aa1239e-321d-4e0e-8183-6fa1361c0fae\") " pod="opendatahub/maas-api-key-cleanup-29686590-6gdcd" Jun 11 16:30:00.313748 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:00.313718 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2scq\" (UniqueName: \"kubernetes.io/projected/0aa1239e-321d-4e0e-8183-6fa1361c0fae-kube-api-access-m2scq\") pod \"maas-api-key-cleanup-29686590-6gdcd\" (UID: \"0aa1239e-321d-4e0e-8183-6fa1361c0fae\") " pod="opendatahub/maas-api-key-cleanup-29686590-6gdcd" Jun 11 16:30:00.471224 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:00.471142 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29686590-6gdcd" Jun 11 16:30:00.612993 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:00.612962 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29686590-6gdcd"] Jun 11 16:30:00.701799 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:00.701765 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29686590-6gdcd" event={"ID":"0aa1239e-321d-4e0e-8183-6fa1361c0fae","Type":"ContainerStarted","Data":"d7748fffe13d7b97240b4d3fc0a6332a06ae9a823f570e4dbd4bf5b924fd9aec"} Jun 11 16:30:11.769337 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:11.769298 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29686590-6gdcd" event={"ID":"0aa1239e-321d-4e0e-8183-6fa1361c0fae","Type":"ContainerStarted","Data":"50178da52c0b3af7474abea97070917dc9c3e54697ea41d2e6e8cc0bc80b996c"} Jun 11 16:30:11.788465 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:11.788409 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29686590-6gdcd" podStartSLOduration=1.054878935 podStartE2EDuration="11.788385685s" totalImagesPullingTime="10.73350675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-11 16:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:30:11.786212184 +0000 UTC m=+1287.620324722" watchObservedRunningTime="2026-06-11 16:30:11.788385685 +0000 UTC m=+1287.622498226" Jun 11 16:30:12.776311 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:12.776273 2573 generic.go:356] "Generic (PLEG): container finished" podID="0aa1239e-321d-4e0e-8183-6fa1361c0fae" containerID="50178da52c0b3af7474abea97070917dc9c3e54697ea41d2e6e8cc0bc80b996c" exitCode=0 Jun 11 16:30:12.776729 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:12.776364 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29686590-6gdcd" event={"ID":"0aa1239e-321d-4e0e-8183-6fa1361c0fae","Type":"ContainerDied","Data":"50178da52c0b3af7474abea97070917dc9c3e54697ea41d2e6e8cc0bc80b996c"} Jun 11 16:30:13.925087 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:13.925055 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29686590-6gdcd" Jun 11 16:30:14.011182 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:14.011141 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m2scq\" (UniqueName: \"kubernetes.io/projected/0aa1239e-321d-4e0e-8183-6fa1361c0fae-kube-api-access-m2scq\") pod \"0aa1239e-321d-4e0e-8183-6fa1361c0fae\" (UID: \"0aa1239e-321d-4e0e-8183-6fa1361c0fae\") " Jun 11 16:30:14.013606 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:14.013576 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0aa1239e-321d-4e0e-8183-6fa1361c0fae-kube-api-access-m2scq" (OuterVolumeSpecName: "kube-api-access-m2scq") pod "0aa1239e-321d-4e0e-8183-6fa1361c0fae" (UID: "0aa1239e-321d-4e0e-8183-6fa1361c0fae"). InnerVolumeSpecName "kube-api-access-m2scq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 11 16:30:14.112361 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:14.112269 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m2scq\" (UniqueName: \"kubernetes.io/projected/0aa1239e-321d-4e0e-8183-6fa1361c0fae-kube-api-access-m2scq\") on node \"ip-10-0-140-111.ec2.internal\" DevicePath \"\"" Jun 11 16:30:14.789339 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:14.789311 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29686590-6gdcd" Jun 11 16:30:14.789517 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:14.789311 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29686590-6gdcd" event={"ID":"0aa1239e-321d-4e0e-8183-6fa1361c0fae","Type":"ContainerDied","Data":"d7748fffe13d7b97240b4d3fc0a6332a06ae9a823f570e4dbd4bf5b924fd9aec"} Jun 11 16:30:14.789517 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:30:14.789443 2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7748fffe13d7b97240b4d3fc0a6332a06ae9a823f570e4dbd4bf5b924fd9aec" Jun 11 16:33:25.434116 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:25.434062 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-758d6cf95f-nkl4c_82b0d29d-8a3e-4840-b049-19fa76dcb173/nginx/0.log" Jun 11 16:33:25.650717 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:25.650686 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-zvhgv_9c3b9f2f-da71-46da-aa84-78734ead79eb/manager/0.log" Jun 11 16:33:25.770068 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:25.769969 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-bm6p8_865d7acf-b0a5-479d-929a-aae71cecab82/manager/0.log" Jun 11 16:33:25.988187 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:25.988159 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29686590-6gdcd_0aa1239e-321d-4e0e-8183-6fa1361c0fae/cleanup/0.log" Jun 11 16:33:26.216565 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:26.216540 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-xmkq6_8ce98183-5dc0-4034-ab51-718722ff4da0/server/0.log" Jun 11 16:33:26.331219 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:26.331183 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-rpwx9_2787e2b3-3de4-400b-962f-2685f017b12e/manager/0.log" Jun 11 16:33:26.439756 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:26.439730 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fb875574b-4skmd_640fbc8e-b66f-4a95-b330-e965be57d616/copy-manifests/0.log" Jun 11 16:33:26.446282 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:26.446250 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fb875574b-4skmd_640fbc8e-b66f-4a95-b330-e965be57d616/manager/0.log" Jun 11 16:33:26.796317 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:26.796281 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-5z6fs_95c045b8-2ba0-4891-bd1b-bcbf849b1686/postgres/0.log" Jun 11 16:33:27.548611 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:27.548586 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j_a8007009-308f-41bb-8554-5e34eb0ebd24/util/0.log" Jun 11 16:33:27.554962 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:27.554944 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j_a8007009-308f-41bb-8554-5e34eb0ebd24/pull/0.log" Jun 11 16:33:27.561228 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:27.561210 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j_a8007009-308f-41bb-8554-5e34eb0ebd24/extract/0.log" Jun 11 16:33:27.678752 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:27.678723 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28_30a58c66-a2a9-4b16-bbad-b6927c94afb4/util/0.log" Jun 11 16:33:27.686812 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:27.686786 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28_30a58c66-a2a9-4b16-bbad-b6927c94afb4/pull/0.log" Jun 11 16:33:27.694323 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:27.694301 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28_30a58c66-a2a9-4b16-bbad-b6927c94afb4/extract/0.log" Jun 11 16:33:27.811601 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:27.811523 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82_071f0967-15ae-4799-a8c3-c31510360054/util/0.log" Jun 11 16:33:27.818732 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:27.818709 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82_071f0967-15ae-4799-a8c3-c31510360054/pull/0.log" Jun 11 16:33:27.826134 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:27.826114 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82_071f0967-15ae-4799-a8c3-c31510360054/extract/0.log" Jun 11 16:33:27.933742 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:27.933716 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc_7366b4af-f894-4c26-8491-e6772f6a723d/util/0.log" Jun 11 16:33:27.939845 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:27.939826 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc_7366b4af-f894-4c26-8491-e6772f6a723d/pull/0.log" Jun 11 16:33:27.945615 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:27.945596 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc_7366b4af-f894-4c26-8491-e6772f6a723d/extract/0.log" Jun 11 16:33:28.293118 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:28.293089 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-q2msk_c5faeb56-6b55-4eaf-be07-d0e90f97f2a4/manager/0.log" Jun 11 16:33:28.856157 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:28.856129 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-fdzzf_0ea44854-c7bb-494b-a38b-3837e18a0cc8/manager/0.log" Jun 11 16:33:29.313077 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:29.313037 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-dns22_e24ed10d-33f8-4bcb-8609-7c04ee8dcf43/discovery/0.log" Jun 11 16:33:29.528756 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:29.528725 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bbbf84858-z99qw_fa0ac318-b430-4185-a0e4-79f600cf3169/kube-auth-proxy/0.log" Jun 11 16:33:29.968185 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:29.968157 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-f5469b655-n2xhl_67e9fb06-c44d-4bd4-bc05-a4df6a246e9d/router/0.log" Jun 11 16:33:30.410655 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:30.410627 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-nfw5p_0fafbdec-54e4-44d5-b7de-ea2e589dc926/storage-initializer/0.log" Jun 11 16:33:30.753065 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:30.752993 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-qgg22_6658471a-3a3a-4085-bdef-d9c19b7c4389/storage-initializer/0.log" Jun 11 16:33:30.759983 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:30.759954 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-qgg22_6658471a-3a3a-4085-bdef-d9c19b7c4389/main/0.log" Jun 11 16:33:37.801297 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:37.801264 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tkm6l_5a412864-1268-4787-be57-d25fd2e7340d/global-pull-secret-syncer/0.log" Jun 11 16:33:37.946205 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:37.946175 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-s6bfw_e9654c3d-7ced-4f3a-8eca-9de0abf92ae3/konnectivity-agent/0.log" Jun 11 16:33:38.030263 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:38.030237 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-111.ec2.internal_3154ce17f76996f6fd8c8d37ce0304d4/haproxy/0.log" Jun 11 16:33:41.768402 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:41.768372 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_servicemesh-operator3-57f65f65fb-9j87l_bcb7bdd4-4abf-47f3-b4a1-3cc7134db49a/sail-operator/0.log" Jun 11 16:33:42.178388 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:42.178357 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j_a8007009-308f-41bb-8554-5e34eb0ebd24/extract/0.log" Jun 11 16:33:42.208054 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:42.208022 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j_a8007009-308f-41bb-8554-5e34eb0ebd24/util/0.log" Jun 11 16:33:42.276206 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:42.276177 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7599r54j_a8007009-308f-41bb-8554-5e34eb0ebd24/pull/0.log" Jun 11 16:33:42.349616 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:42.349592 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28_30a58c66-a2a9-4b16-bbad-b6927c94afb4/extract/0.log" Jun 11 16:33:42.414021 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:42.413995 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28_30a58c66-a2a9-4b16-bbad-b6927c94afb4/util/0.log" Jun 11 16:33:42.467426 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:42.467334 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0c9m28_30a58c66-a2a9-4b16-bbad-b6927c94afb4/pull/0.log" Jun 11 16:33:42.505288 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:42.505250 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82_071f0967-15ae-4799-a8c3-c31510360054/extract/0.log" Jun 11 16:33:42.541731 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:42.541704 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82_071f0967-15ae-4799-a8c3-c31510360054/util/0.log" Jun 11 16:33:42.577309 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:42.577280 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed734jp82_071f0967-15ae-4799-a8c3-c31510360054/pull/0.log" Jun 11 16:33:42.620092 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:42.620056 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc_7366b4af-f894-4c26-8491-e6772f6a723d/extract/0.log" Jun 11 16:33:42.646798 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:42.646769 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc_7366b4af-f894-4c26-8491-e6772f6a723d/util/0.log" Jun 11 16:33:42.671606 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:42.671579 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1xhhpc_7366b4af-f894-4c26-8491-e6772f6a723d/pull/0.log" Jun 11 16:33:42.770703 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:42.770605 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-q2msk_c5faeb56-6b55-4eaf-be07-d0e90f97f2a4/manager/0.log" Jun 11 16:33:43.003692 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:43.003658 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-fdzzf_0ea44854-c7bb-494b-a38b-3837e18a0cc8/manager/0.log" Jun 11 16:33:44.665165 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:44.665140 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p7nvz_7acea643-2978-4893-a6a9-bf88a11b531b/ovn-acl-logging/0.log" Jun 11 16:33:44.672739 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:44.672713 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-p7nvz_7acea643-2978-4893-a6a9-bf88a11b531b/ovn-acl-logging/0.log" Jun 11 16:33:44.843866 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:44.843842 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-wk9p8_927f7de8-cf3a-48da-9597-bb088f8ee605/kube-state-metrics/0.log" Jun 11 16:33:44.868845 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:44.868819 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-wk9p8_927f7de8-cf3a-48da-9597-bb088f8ee605/kube-rbac-proxy-main/0.log" Jun 11 16:33:44.895184 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:44.895155 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-57bbf8bfb5-wk9p8_927f7de8-cf3a-48da-9597-bb088f8ee605/kube-rbac-proxy-self/0.log" Jun 11 16:33:44.977239 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:44.977144 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-47k7t_f797967f-8904-4f56-98bd-92dfb9eb5143/monitoring-plugin/0.log" Jun 11 16:33:45.213163 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:45.213137 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f4t65_4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2/node-exporter/0.log" Jun 11 16:33:45.238328 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:45.238254 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f4t65_4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2/kube-rbac-proxy/0.log" Jun 11 16:33:45.259892 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:45.259867 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f4t65_4b2e3c1f-e706-4991-bbc8-eb09cbc3ddd2/init-textfile/0.log" Jun 11 16:33:45.545616 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:45.545531 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-8cnxs_7887f863-9af9-45a9-9d02-73da43f4aff2/prometheus-operator/0.log" Jun 11 16:33:45.566286 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:45.566262 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-8cnxs_7887f863-9af9-45a9-9d02-73da43f4aff2/kube-rbac-proxy/0.log" Jun 11 16:33:45.798773 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:45.798693 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5ff6dfb9c5-lr8wd_3d778107-2764-41eb-ba2f-e1042dcc3c5a/thanos-query/0.log" Jun 11 16:33:45.825395 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:45.825372 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5ff6dfb9c5-lr8wd_3d778107-2764-41eb-ba2f-e1042dcc3c5a/kube-rbac-proxy-web/0.log" Jun 11 16:33:45.850295 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:45.850271 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5ff6dfb9c5-lr8wd_3d778107-2764-41eb-ba2f-e1042dcc3c5a/kube-rbac-proxy/0.log" Jun 11 16:33:45.874297 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:45.874272 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5ff6dfb9c5-lr8wd_3d778107-2764-41eb-ba2f-e1042dcc3c5a/prom-label-proxy/0.log" Jun 11 16:33:45.898884 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:45.898857 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5ff6dfb9c5-lr8wd_3d778107-2764-41eb-ba2f-e1042dcc3c5a/kube-rbac-proxy-rules/0.log" Jun 11 16:33:45.925053 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:45.925027 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5ff6dfb9c5-lr8wd_3d778107-2764-41eb-ba2f-e1042dcc3c5a/kube-rbac-proxy-metrics/0.log" Jun 11 16:33:48.200372 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.200320 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf"] Jun 11 16:33:48.200855 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.200837 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0aa1239e-321d-4e0e-8183-6fa1361c0fae" containerName="cleanup" Jun 11 16:33:48.200973 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.200859 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0aa1239e-321d-4e0e-8183-6fa1361c0fae" containerName="cleanup" Jun 11 16:33:48.201048 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.200974 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0aa1239e-321d-4e0e-8183-6fa1361c0fae" containerName="cleanup" Jun 11 16:33:48.204279 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.204262 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:48.206282 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.206264 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fvx9r\"/\"openshift-service-ca.crt\"" Jun 11 16:33:48.206911 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.206896 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fvx9r\"/\"default-dockercfg-xmzhk\"" Jun 11 16:33:48.207008 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.206901 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fvx9r\"/\"kube-root-ca.crt\"" Jun 11 16:33:48.213555 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.213531 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf"] Jun 11 16:33:48.287666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.287629 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95a50304-94e7-4852-b837-89a0b9930f1f-sys\") pod \"perf-node-gather-daemonset-jl4qf\" (UID: \"95a50304-94e7-4852-b837-89a0b9930f1f\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:48.287666 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.287672 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/95a50304-94e7-4852-b837-89a0b9930f1f-podres\") pod \"perf-node-gather-daemonset-jl4qf\" (UID: \"95a50304-94e7-4852-b837-89a0b9930f1f\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:48.287884 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.287691 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpdpn\" (UniqueName: \"kubernetes.io/projected/95a50304-94e7-4852-b837-89a0b9930f1f-kube-api-access-hpdpn\") pod \"perf-node-gather-daemonset-jl4qf\" (UID: \"95a50304-94e7-4852-b837-89a0b9930f1f\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:48.287884 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.287767 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/95a50304-94e7-4852-b837-89a0b9930f1f-proc\") pod \"perf-node-gather-daemonset-jl4qf\" (UID: \"95a50304-94e7-4852-b837-89a0b9930f1f\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:48.287884 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.287872 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95a50304-94e7-4852-b837-89a0b9930f1f-lib-modules\") pod \"perf-node-gather-daemonset-jl4qf\" (UID: \"95a50304-94e7-4852-b837-89a0b9930f1f\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:48.389087 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.389047 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/95a50304-94e7-4852-b837-89a0b9930f1f-podres\") pod \"perf-node-gather-daemonset-jl4qf\" (UID: \"95a50304-94e7-4852-b837-89a0b9930f1f\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:48.389087 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.389090 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpdpn\" (UniqueName: \"kubernetes.io/projected/95a50304-94e7-4852-b837-89a0b9930f1f-kube-api-access-hpdpn\") pod \"perf-node-gather-daemonset-jl4qf\" (UID: \"95a50304-94e7-4852-b837-89a0b9930f1f\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:48.389268 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.389215 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/95a50304-94e7-4852-b837-89a0b9930f1f-proc\") pod \"perf-node-gather-daemonset-jl4qf\" (UID: \"95a50304-94e7-4852-b837-89a0b9930f1f\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:48.389268 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.389238 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/95a50304-94e7-4852-b837-89a0b9930f1f-podres\") pod \"perf-node-gather-daemonset-jl4qf\" (UID: \"95a50304-94e7-4852-b837-89a0b9930f1f\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:48.389339 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.389314 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/95a50304-94e7-4852-b837-89a0b9930f1f-proc\") pod \"perf-node-gather-daemonset-jl4qf\" (UID: \"95a50304-94e7-4852-b837-89a0b9930f1f\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:48.389374 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.389317 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95a50304-94e7-4852-b837-89a0b9930f1f-lib-modules\") pod \"perf-node-gather-daemonset-jl4qf\" (UID: \"95a50304-94e7-4852-b837-89a0b9930f1f\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:48.389411 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.389396 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95a50304-94e7-4852-b837-89a0b9930f1f-sys\") pod \"perf-node-gather-daemonset-jl4qf\" (UID: \"95a50304-94e7-4852-b837-89a0b9930f1f\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:48.389411 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.389401 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/95a50304-94e7-4852-b837-89a0b9930f1f-lib-modules\") pod \"perf-node-gather-daemonset-jl4qf\" (UID: \"95a50304-94e7-4852-b837-89a0b9930f1f\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:48.389476 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.389436 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/95a50304-94e7-4852-b837-89a0b9930f1f-sys\") pod \"perf-node-gather-daemonset-jl4qf\" (UID: \"95a50304-94e7-4852-b837-89a0b9930f1f\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:48.400855 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.400831 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpdpn\" (UniqueName: \"kubernetes.io/projected/95a50304-94e7-4852-b837-89a0b9930f1f-kube-api-access-hpdpn\") pod \"perf-node-gather-daemonset-jl4qf\" (UID: \"95a50304-94e7-4852-b837-89a0b9930f1f\") " pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:48.515313 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.515236 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:48.541548 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.541512 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6657875f97-9s44r_f99600e4-7f1f-4814-b30b-3a82984c46c6/download-server/0.log" Jun 11 16:33:48.656226 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:48.656193 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf"] Jun 11 16:33:48.657197 ip-10-0-140-111 kubenswrapper[2573]: W0611 16:33:48.657169 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod95a50304_94e7_4852_b837_89a0b9930f1f.slice/crio-7026bca787f4871a36031c7903e4c387a814504ef6c69f455aa615ab70a787e4 WatchSource:0}: Error finding container 7026bca787f4871a36031c7903e4c387a814504ef6c69f455aa615ab70a787e4: Status 404 returned error can't find the container with id 7026bca787f4871a36031c7903e4c387a814504ef6c69f455aa615ab70a787e4 Jun 11 16:33:49.026279 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:49.026241 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" event={"ID":"95a50304-94e7-4852-b837-89a0b9930f1f","Type":"ContainerStarted","Data":"926572a935b1d50281fd838570934643c2eae9fc4f687abe73bc0587ae57de7a"} Jun 11 16:33:49.026498 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:49.026298 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" event={"ID":"95a50304-94e7-4852-b837-89a0b9930f1f","Type":"ContainerStarted","Data":"7026bca787f4871a36031c7903e4c387a814504ef6c69f455aa615ab70a787e4"} Jun 11 16:33:49.026498 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:49.026385 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:49.042333 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:49.042282 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" podStartSLOduration=1.042266424 podStartE2EDuration="1.042266424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-11 16:33:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-11 16:33:49.040823657 +0000 UTC m=+1504.874936188" watchObservedRunningTime="2026-06-11 16:33:49.042266424 +0000 UTC m=+1504.876378962" Jun 11 16:33:50.533464 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:50.533430 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xwb7k_593fc31d-18b7-4f07-b658-720932870bf6/dns/0.log" Jun 11 16:33:50.557328 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:50.557302 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xwb7k_593fc31d-18b7-4f07-b658-720932870bf6/kube-rbac-proxy/0.log" Jun 11 16:33:50.665144 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:50.665108 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k25qj_85d4cd35-2225-4a31-b023-336e1ccbbdc7/dns-node-resolver/0.log" Jun 11 16:33:51.327091 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:51.327063 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gmrpd_17c6b2e4-8351-4bab-a83a-53b33503505d/node-ca/0.log" Jun 11 16:33:52.430167 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:52.430135 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-dns22_e24ed10d-33f8-4bcb-8609-7c04ee8dcf43/discovery/0.log" Jun 11 16:33:52.478165 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:52.478133 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bbbf84858-z99qw_fa0ac318-b430-4185-a0e4-79f600cf3169/kube-auth-proxy/0.log" Jun 11 16:33:52.583793 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:52.583763 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-f5469b655-n2xhl_67e9fb06-c44d-4bd4-bc05-a4df6a246e9d/router/0.log" Jun 11 16:33:53.258710 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:53.258676 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g8957_b4ff7d18-6726-42f5-9f39-a36097a40fb2/serve-healthcheck-canary/0.log" Jun 11 16:33:53.902150 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:53.902118 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bt9lv_729d8eef-e25d-44e9-a4e8-df71a5ba6b7d/kube-rbac-proxy/0.log" Jun 11 16:33:53.923654 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:53.923622 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bt9lv_729d8eef-e25d-44e9-a4e8-df71a5ba6b7d/exporter/0.log" Jun 11 16:33:53.945718 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:53.945685 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bt9lv_729d8eef-e25d-44e9-a4e8-df71a5ba6b7d/extractor/0.log" Jun 11 16:33:55.043128 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:55.043104 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fvx9r/perf-node-gather-daemonset-jl4qf" Jun 11 16:33:56.457018 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:56.456973 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-758d6cf95f-nkl4c_82b0d29d-8a3e-4840-b049-19fa76dcb173/nginx/0.log" Jun 11 16:33:56.513434 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:56.513387 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-zvhgv_9c3b9f2f-da71-46da-aa84-78734ead79eb/manager/0.log" Jun 11 16:33:56.555435 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:56.555401 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-bm6p8_865d7acf-b0a5-479d-929a-aae71cecab82/manager/0.log" Jun 11 16:33:56.605648 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:56.605619 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29686590-6gdcd_0aa1239e-321d-4e0e-8183-6fa1361c0fae/cleanup/0.log" Jun 11 16:33:56.673656 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:56.673630 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-xmkq6_8ce98183-5dc0-4034-ab51-718722ff4da0/server/0.log" Jun 11 16:33:56.740454 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:56.740365 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-rpwx9_2787e2b3-3de4-400b-962f-2685f017b12e/manager/0.log" Jun 11 16:33:56.773882 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:56.773855 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fb875574b-4skmd_640fbc8e-b66f-4a95-b330-e965be57d616/manager/0.log" Jun 11 16:33:56.798491 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:56.798465 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-fb875574b-4skmd_640fbc8e-b66f-4a95-b330-e965be57d616/copy-manifests/0.log" Jun 11 16:33:57.004813 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:33:57.004723 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-5z6fs_95c045b8-2ba0-4891-bd1b-bcbf849b1686/postgres/0.log" Jun 11 16:34:02.903332 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:34:02.903259 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-dns22_e24ed10d-33f8-4bcb-8609-7c04ee8dcf43/discovery/0.log" Jun 11 16:34:02.928852 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:34:02.928828 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bbbf84858-z99qw_fa0ac318-b430-4185-a0e4-79f600cf3169/kube-auth-proxy/0.log" Jun 11 16:34:03.000064 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:34:03.000033 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-f5469b655-n2xhl_67e9fb06-c44d-4bd4-bc05-a4df6a246e9d/router/0.log" Jun 11 16:34:04.192064 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:34:04.192036 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-dns22_e24ed10d-33f8-4bcb-8609-7c04ee8dcf43/discovery/0.log" Jun 11 16:34:04.218909 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:34:04.218868 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6bbbf84858-z99qw_fa0ac318-b430-4185-a0e4-79f600cf3169/kube-auth-proxy/0.log" Jun 11 16:34:04.287673 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:34:04.287645 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-f5469b655-n2xhl_67e9fb06-c44d-4bd4-bc05-a4df6a246e9d/router/0.log" Jun 11 16:34:04.399428 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:34:04.399388 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-4ldr7_f2379c57-3673-4fbd-84a8-936789b26e10/migrator/0.log" Jun 11 16:34:04.423302 ip-10-0-140-111 kubenswrapper[2573]: I0611 16:34:04.423274 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-955ffdf5f-4ldr7_f2379c57-3673-4fbd-84a8-936789b26e10/graceful-termination/0.log"