Jul 08 17:14:19.940636 ip-10-0-128-50 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 17:14:19.940651 ip-10-0-128-50 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 17:14:19.940660 ip-10-0-128-50 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 17:14:19.941059 ip-10-0-128-50 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 17:14:30.051178 ip-10-0-128-50 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 17:14:30.051196 ip-10-0-128-50 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e7b5d6b4635b44fe9ad930a09970e7ee -- Jul 08 17:16:04.037321 ip-10-0-128-50 systemd[1]: Starting Kubernetes Kubelet... Jul 08 17:16:04.471320 ip-10-0-128-50 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 17:16:04.471320 ip-10-0-128-50 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 17:16:04.471320 ip-10-0-128-50 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 17:16:04.471320 ip-10-0-128-50 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 17:16:04.471320 ip-10-0-128-50 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 17:16:04.471320 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.464612 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472065 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472082 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472087 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472091 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472094 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472096 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472099 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472102 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472104 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472107 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472110 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472112 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472115 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472118 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472121 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472123 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472126 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472130 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:16:04.487487 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472132 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472135 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472137 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472140 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472142 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472145 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472148 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472151 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472154 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472157 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472160 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472163 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472165 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472175 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472178 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472181 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472183 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472186 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472189 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472192 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:16:04.502154 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472194 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472197 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472200 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472203 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472207 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472210 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472213 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472216 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472219 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472222 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472225 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472227 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472230 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472235 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472238 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472241 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472243 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472246 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472249 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:16:04.504195 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472251 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472254 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472256 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472259 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472261 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472264 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472266 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472270 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472272 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472275 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472278 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472281 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472284 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472286 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472288 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472291 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472294 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472296 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472299 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472302 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:16:04.506529 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472304 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472307 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472309 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472312 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472315 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472317 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472323 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472326 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472328 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472331 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472333 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472337 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472339 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472342 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472344 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472347 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472349 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472352 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472354 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472357 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:16:04.548465 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472360 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.472362 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473102 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473110 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473113 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473116 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473118 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473121 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473123 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473126 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473128 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473131 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473134 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473137 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473140 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473143 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473145 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473148 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473153 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473156 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:16:04.569066 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473158 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:16:04.550979 ip-10-0-128-50 systemd[1]: Started Kubernetes Kubelet. Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473161 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473164 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473166 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473169 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473172 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473175 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473177 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473180 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473183 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473185 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473187 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473190 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473193 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473196 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473198 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473201 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473204 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473206 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473208 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:16:04.811200 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473211 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473213 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473216 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473218 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473221 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473223 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473225 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473228 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473230 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473235 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473239 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473241 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473245 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473247 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473250 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473253 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473257 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473261 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473264 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473267 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:16:04.813615 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473270 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473272 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473275 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473278 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473280 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473283 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473285 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473288 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473290 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473295 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473297 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473300 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473302 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473305 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473307 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473309 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473312 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473314 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473317 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473319 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:16:04.814378 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473321 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473327 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473331 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473334 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473338 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473342 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473345 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473348 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473351 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473353 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473356 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473359 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473361 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473364 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473366 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473369 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473371 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473374 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473378 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:16:04.815074 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473380 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473382 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473761 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473766 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473769 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473772 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473774 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473777 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473779 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473782 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473785 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473787 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473790 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473792 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473796 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473800 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473803 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473805 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473808 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473811 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:16:04.815626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473813 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473816 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473818 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473821 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473824 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473826 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473829 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473832 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473834 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473837 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473840 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473842 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473845 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473853 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473856 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473858 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473861 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473863 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473866 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:16:04.816239 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473868 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473871 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473873 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473876 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473878 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473880 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473883 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473887 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473891 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473893 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473896 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473898 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473901 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473904 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473906 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473908 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473911 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473914 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473916 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473918 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:16:04.816786 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473921 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473923 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473926 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473928 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473931 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473933 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473935 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473938 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473940 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473943 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473946 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473948 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473951 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473953 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473956 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473958 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473961 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.473963 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474851 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474859 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:16:04.817390 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474865 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474868 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474871 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474874 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474878 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474882 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474885 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474889 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474892 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474895 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474897 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474900 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474903 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474905 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474908 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474910 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474913 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474915 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474918 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:16:04.817940 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474921 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474923 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.474927 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.474999 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475029 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475039 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475046 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475050 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475053 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475058 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475062 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475065 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475069 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475074 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475077 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475081 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475084 2578 flags.go:64] FLAG: --cgroup-root="" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475087 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475090 2578 flags.go:64] FLAG: --client-ca-file="" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475097 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475100 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475105 2578 flags.go:64] FLAG: --cluster-domain="" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475108 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 17:16:04.818535 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475111 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475114 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475117 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475121 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475124 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475127 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475130 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475133 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475136 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475139 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475142 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475145 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475150 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475153 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475156 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475158 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475161 2578 flags.go:64] FLAG: --enable-server="true" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475164 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475169 2578 flags.go:64] FLAG: --event-burst="100" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475172 2578 flags.go:64] FLAG: --event-qps="50" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475175 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475180 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475184 2578 flags.go:64] FLAG: --eviction-hard="" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475189 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 17:16:04.819199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475191 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475194 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475197 2578 flags.go:64] FLAG: --eviction-soft="" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475200 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475203 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475207 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475210 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475212 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475215 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475218 2578 flags.go:64] FLAG: --feature-gates="" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475222 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475225 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475228 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475231 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475234 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475237 2578 flags.go:64] FLAG: --help="false" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475239 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-128-50.ec2.internal" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475243 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475245 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475248 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475252 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475255 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475258 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 17:16:04.819926 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475261 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475264 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475266 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475269 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475272 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475275 2578 flags.go:64] FLAG: --kube-reserved="" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475280 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475284 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475287 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475290 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475292 2578 flags.go:64] FLAG: --lock-file="" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475295 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475298 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475301 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475306 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475310 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475312 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475315 2578 flags.go:64] FLAG: --logging-format="text" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475318 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475321 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475324 2578 flags.go:64] FLAG: --manifest-url="" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475327 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475331 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475334 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475338 2578 flags.go:64] FLAG: --max-pods="110" Jul 08 17:16:04.820635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475341 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475344 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475347 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475350 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475352 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475355 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475358 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475365 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475369 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475372 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475375 2578 flags.go:64] FLAG: --pod-cidr="" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475377 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475383 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475386 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475392 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475395 2578 flags.go:64] FLAG: --port="10250" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475398 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475401 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-086dfd9fcd1a70a2b" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475404 2578 flags.go:64] FLAG: --qos-reserved="" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475407 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475410 2578 flags.go:64] FLAG: --register-node="true" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475413 2578 flags.go:64] FLAG: --register-with-taints="" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475416 2578 flags.go:64] FLAG: --registry-burst="10" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475420 2578 flags.go:64] FLAG: --registry-qps="5" Jul 08 17:16:04.821345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475423 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475425 2578 flags.go:64] FLAG: --reserved-memory="" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475429 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475432 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475435 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475438 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475441 2578 flags.go:64] FLAG: --runonce="false" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475443 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475446 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475449 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475452 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475455 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475458 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475461 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475464 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475467 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475470 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475473 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475475 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475478 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475482 2578 flags.go:64] FLAG: --system-cgroups="" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475484 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475493 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475496 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475499 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 17:16:04.821960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475503 2578 flags.go:64] FLAG: --tls-min-version="" Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475506 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475509 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475511 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475514 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475517 2578 flags.go:64] FLAG: --v="2" Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475521 2578 flags.go:64] FLAG: --version="false" Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475525 2578 flags.go:64] FLAG: --vmodule="" Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475532 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.475535 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475650 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475654 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475657 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475660 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475663 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475666 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475669 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475672 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475676 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475680 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475683 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475686 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:16:04.822660 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475688 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475691 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475693 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475696 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475699 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475702 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475704 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475709 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475712 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475715 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475718 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475720 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475723 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475726 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475728 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475731 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475734 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475736 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475739 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475742 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:16:04.823263 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475745 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475748 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475750 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475753 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475755 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475758 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475760 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475763 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475765 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475768 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475771 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475773 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475776 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475778 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475781 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475784 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475786 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475789 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475791 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475796 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:16:04.823820 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475799 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475802 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475804 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475807 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475809 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475812 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475815 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475818 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475820 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475822 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475825 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475828 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475831 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475833 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475836 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475838 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475841 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475844 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475846 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:16:04.824442 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475848 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475851 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475854 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475856 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475859 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475861 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475864 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475866 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475869 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475871 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475874 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475876 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475881 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475884 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475887 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475889 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475893 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475897 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475900 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:16:04.824995 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475902 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:16:04.827922 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475905 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:16:04.827922 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475908 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:16:04.827922 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475911 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:16:04.827922 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475913 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:16:04.827922 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475916 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:16:04.827922 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475920 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:16:04.827922 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475923 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:16:04.827922 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.475926 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:16:04.827922 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.476709 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 17:16:04.827922 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.483743 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 17:16:04.827922 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.483758 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 17:16:04.827922 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.483778 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 17:16:04.827922 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.483783 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 17:16:04.827922 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483837 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483842 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483845 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483848 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483851 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483854 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483857 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483860 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483863 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483865 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483868 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483870 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483873 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483876 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483879 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483881 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483884 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483887 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483889 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483892 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:16:04.828534 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483894 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483897 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483900 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483903 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483906 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483909 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483911 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483914 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483916 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483920 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483922 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483925 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483928 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483930 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483933 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483935 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483938 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483940 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483943 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483946 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:16:05.001403 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483949 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483953 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483957 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483960 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483963 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483966 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483969 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483972 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483974 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483977 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483980 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483983 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483985 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483989 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483991 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483994 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.483998 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484001 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484004 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:16:05.002086 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484020 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484024 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484027 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484030 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484033 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484035 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484038 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484041 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484043 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484046 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484049 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484051 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484057 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484060 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484063 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484065 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484068 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484071 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484074 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484077 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:16:05.002637 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484079 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484082 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484084 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484087 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484090 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484092 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484095 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484097 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484100 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484103 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484106 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484108 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484111 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484113 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484116 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484118 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484121 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484124 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484127 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:16:05.003212 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484129 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:16:05.003710 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.484134 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 17:16:05.003710 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484242 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:16:05.003710 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484248 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:16:05.003710 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484253 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:16:05.003710 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484256 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:16:05.003710 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484260 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:16:05.003710 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484263 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:16:05.003710 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484266 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:16:05.003710 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484269 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:16:05.003710 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484271 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:16:05.003710 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484274 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:16:05.003710 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484277 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:16:05.003710 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484280 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:16:05.003710 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484283 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484286 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484289 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484291 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484294 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484297 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484299 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484302 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484304 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484307 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484309 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484312 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484314 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484317 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484319 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484322 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484324 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484327 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484329 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484332 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:16:05.004149 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484334 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484337 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484339 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484342 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484345 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484348 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484351 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484353 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484356 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484358 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484360 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484363 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484365 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484369 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484372 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484374 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484377 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484379 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484382 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484384 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:16:05.004708 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484387 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484389 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484392 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484394 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484397 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484399 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484402 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484404 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484407 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484410 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484413 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484415 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484417 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484421 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484424 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484427 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484431 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484434 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484437 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:16:05.005293 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484440 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484443 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484445 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484448 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484450 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484453 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484457 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484459 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484462 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484465 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484467 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484470 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484472 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484475 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484477 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484480 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484482 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484485 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484488 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484491 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:16:05.005781 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484494 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:16:05.006396 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484497 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:16:05.006396 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484499 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:16:05.006396 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484502 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:16:05.006396 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484505 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:16:05.006396 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484507 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:16:05.006396 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484510 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:16:05.006396 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:04.484512 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:16:05.006396 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.484517 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 17:16:05.006396 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.484639 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 17:16:05.006396 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.486566 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 17:16:05.006396 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.488819 2578 server.go:1019] "Starting client certificate rotation" Jul 08 17:16:05.006396 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.488923 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 17:16:05.006396 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.488964 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 17:16:05.006764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.491232 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 17:16:05.006764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.498038 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 17:16:05.006764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.514668 2578 log.go:25] "Validated CRI v1 runtime API" Jul 08 17:16:05.006764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.517561 2578 log.go:25] "Validated CRI v1 image API" Jul 08 17:16:05.006764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.519581 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 17:16:05.006764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.523994 2578 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ab5f69ce-a4cc-4c20-9f4b-e0330974c585:/dev/nvme0n1p4 dc9e755a-00ff-4ca7-9f06-68c4cb0c56d4:/dev/nvme0n1p3] Jul 08 17:16:05.006764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.524036 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 17:16:05.006764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.527646 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.529898 2578 manager.go:216] Machine: {Timestamp:2026-07-08 17:16:04.528534771 +0000 UTC m=+0.357089128 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101010 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28476a3b0ae67fc767f61290f077c5 SystemUUID:ec28476a-3b0a-e67f-c767-f61290f077c5 BootID:e7b5d6b4-635b-44fe-9ad9-30a09970e7ee Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d3:e1:84:b0:65 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d3:e1:84:b0:65 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:f9:96:e7:f6:8d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.530004 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.530093 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.533044 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.533066 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-50.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.534130 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.534142 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.534149 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.534738 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.535966 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.539723 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.539857 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.539884 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.539894 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.539909 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.539920 2578 kubelet.go:398] "Adding apiserver pod source" Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.539929 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 17:16:05.007097 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.540228 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.540248 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.542228 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.545440 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.545469 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.546474 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.546488 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.546494 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.546500 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.546510 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.546516 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.546524 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.546529 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.546536 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.546542 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.546550 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.546555 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.547260 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.547267 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.550195 2578 server.go:1268] "Started kubelet" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.550277 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.550281 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.550352 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.552040 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.552083 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.553337 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-50.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.553835 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-50.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.553836 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.555901 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.557489 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.557500 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.557608 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.557943 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.557957 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.557971 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.558234 2578 factory.go:55] Registering systemd factory Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.558269 2578 factory.go:223] Registration of the systemd container factory successfully Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.558324 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.558334 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.558496 2578 factory.go:153] Registering CRI-O factory Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.558510 2578 factory.go:223] Registration of the crio container factory successfully Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.558573 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.558597 2578 factory.go:103] Registering Raw factory Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.558611 2578 manager.go:1196] Started watching for new ooms in manager Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.558997 2578 manager.go:349] Starting recovery of all containers Jul 08 17:16:05.007685 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.560668 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.568990 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-50.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.560690 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-50.ec2.internal.18c05ff673ec36f3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-50.ec2.internal,UID:ip-10-0-128-50.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-50.ec2.internal,},FirstTimestamp:2026-07-08 17:16:04.550170355 +0000 UTC m=+0.378724690,LastTimestamp:2026-07-08 17:16:04.550170355 +0000 UTC m=+0.378724690,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-50.ec2.internal,}" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.573962 2578 manager.go:354] Recovery completed Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.578697 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.580738 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.580770 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.580784 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.581262 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.581270 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.581276 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.581284 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.581955 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.581978 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.581992 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.583038 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-50.ec2.internal.18c05ff675bee137 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-50.ec2.internal,UID:ip-10-0-128-50.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-50.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-50.ec2.internal,},FirstTimestamp:2026-07-08 17:16:04.580753719 +0000 UTC m=+0.409308056,LastTimestamp:2026-07-08 17:16:04.580753719 +0000 UTC m=+0.409308056,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-50.ec2.internal,}" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.583418 2578 policy_none.go:49] "None policy: Start" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.583430 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.583440 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.585333 2578 policy_none.go:47] "Start" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.596790 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-50.ec2.internal.18c05ff675bf39c1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-50.ec2.internal,UID:ip-10-0-128-50.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-50.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-50.ec2.internal,},FirstTimestamp:2026-07-08 17:16:04.580776385 +0000 UTC m=+0.409330722,LastTimestamp:2026-07-08 17:16:04.580776385 +0000 UTC m=+0.409330722,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-50.ec2.internal,}" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.602974 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vsvnn" Jul 08 17:16:05.008853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.614947 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vsvnn" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.615489 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-50.ec2.internal.18c05ff675bf6a05 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-50.ec2.internal,UID:ip-10-0-128-50.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-50.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-50.ec2.internal,},FirstTimestamp:2026-07-08 17:16:04.580788741 +0000 UTC m=+0.409343078,LastTimestamp:2026-07-08 17:16:04.580788741 +0000 UTC m=+0.409343078,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-50.ec2.internal,}" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.651553 2578 manager.go:337] "Starting Device Plugin manager" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.651623 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.651635 2578 server.go:87] "Starting device plugin registration server" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.651891 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.651904 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.652025 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.652107 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.652406 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.652631 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.652675 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-50.ec2.internal\" not found" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.752870 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.753910 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.753945 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.753955 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.753981 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-50.ec2.internal" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.773031 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-50.ec2.internal" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.773055 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-50.ec2.internal\": node \"ip-10-0-128-50.ec2.internal\" not found" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.783764 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.784996 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.785048 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.785076 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.785117 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.797758 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.879689 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.885754 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal"] Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.885815 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.886775 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.886805 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.886814 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.889208 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.889351 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.889376 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.889968 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.889993 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.890002 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:16:05.009492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.889970 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:16:05.010509 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.890041 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:16:05.010509 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.890059 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:16:05.010509 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.892195 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Jul 08 17:16:05.010509 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.892214 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:16:05.010509 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.892893 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:16:05.010509 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.892931 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:16:05.010509 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.892944 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:16:05.010509 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.920572 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-50.ec2.internal\" not found" node="ip-10-0-128-50.ec2.internal" Jul 08 17:16:05.010509 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.924117 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-50.ec2.internal\" not found" node="ip-10-0-128-50.ec2.internal" Jul 08 17:16:05.010509 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.960023 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/409dd90d59cc55e7490a78f8e6f69617-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"409dd90d59cc55e7490a78f8e6f69617\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 08 17:16:05.010509 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.960046 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/04a30fea7d8cb69c63b01d701b00e3be-config\") pod \"kube-apiserver-proxy-ip-10-0-128-50.ec2.internal\" (UID: \"04a30fea7d8cb69c63b01d701b00e3be\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Jul 08 17:16:05.010509 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:04.960063 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/409dd90d59cc55e7490a78f8e6f69617-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"409dd90d59cc55e7490a78f8e6f69617\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 08 17:16:05.010509 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:04.980344 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Jul 08 17:16:05.177799 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:05.060876 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/409dd90d59cc55e7490a78f8e6f69617-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"409dd90d59cc55e7490a78f8e6f69617\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 08 17:16:05.177799 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:05.060902 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/409dd90d59cc55e7490a78f8e6f69617-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"409dd90d59cc55e7490a78f8e6f69617\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 08 17:16:05.177799 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:05.060918 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/04a30fea7d8cb69c63b01d701b00e3be-config\") pod \"kube-apiserver-proxy-ip-10-0-128-50.ec2.internal\" (UID: \"04a30fea7d8cb69c63b01d701b00e3be\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Jul 08 17:16:05.177799 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:05.060947 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/04a30fea7d8cb69c63b01d701b00e3be-config\") pod \"kube-apiserver-proxy-ip-10-0-128-50.ec2.internal\" (UID: \"04a30fea7d8cb69c63b01d701b00e3be\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Jul 08 17:16:05.177799 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:05.060964 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/409dd90d59cc55e7490a78f8e6f69617-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"409dd90d59cc55e7490a78f8e6f69617\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 08 17:16:05.177799 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:05.060974 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/409dd90d59cc55e7490a78f8e6f69617-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal\" (UID: \"409dd90d59cc55e7490a78f8e6f69617\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 08 17:16:05.177799 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:05.080813 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Jul 08 17:16:05.323605 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:05.181083 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Jul 08 17:16:05.323605 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:05.224254 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 08 17:16:05.323605 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:05.227380 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Jul 08 17:16:05.323605 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:05.281155 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Jul 08 17:16:05.466883 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:05.381639 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-50.ec2.internal\" not found" Jul 08 17:16:05.467133 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:05.466998 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:05.487842 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:05.487983 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:05.488004 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:05.505409 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.143005 2578 apiserver.go:52] "Watching apiserver" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.156370 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.156655 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-8dk7z","openshift-network-operator/iptables-alerter-m8fbb","openshift-ovn-kubernetes/ovnkube-node-jk4p4","kube-system/konnectivity-agent-v2p4n","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt","openshift-cluster-node-tuning-operator/tuned-586wx","openshift-dns/node-resolver-xntds","openshift-image-registry/node-ca-7555c","openshift-multus/multus-9wmcv","openshift-multus/multus-additional-cni-plugins-2hjbv","openshift-network-diagnostics/network-check-target-7n9w9","kube-system/global-pull-secret-syncer-pqz5t"] Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.159086 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.159159 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.160285 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.160300 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.161119 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m8fbb" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.163500 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.164632 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.165825 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs\") pod \"network-metrics-daemon-8dk7z\" (UID: \"78604c5e-b53b-4761-b62e-b7f964af3e44\") " pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.165844 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8g85\" (UniqueName: \"kubernetes.io/projected/78604c5e-b53b-4761-b62e-b7f964af3e44-kube-api-access-n8g85\") pod \"network-metrics-daemon-8dk7z\" (UID: \"78604c5e-b53b-4761-b62e-b7f964af3e44\") " pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.165860 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ed994a80-13df-422c-be49-efca7eebe7d7-iptables-alerter-script\") pod \"iptables-alerter-m8fbb\" (UID: \"ed994a80-13df-422c-be49-efca7eebe7d7\") " pod="openshift-network-operator/iptables-alerter-m8fbb" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.165874 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed994a80-13df-422c-be49-efca7eebe7d7-host-slash\") pod \"iptables-alerter-m8fbb\" (UID: \"ed994a80-13df-422c-be49-efca7eebe7d7\") " pod="openshift-network-operator/iptables-alerter-m8fbb" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.165891 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgh7f\" (UniqueName: \"kubernetes.io/projected/ed994a80-13df-422c-be49-efca7eebe7d7-kube-api-access-mgh7f\") pod \"iptables-alerter-m8fbb\" (UID: \"ed994a80-13df-422c-be49-efca7eebe7d7\") " pod="openshift-network-operator/iptables-alerter-m8fbb" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.165986 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v2p4n" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.168056 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.169303 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.171636 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xntds" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.172842 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.172854 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 17:16:07.182699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.173107 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zwx7r\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.173131 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.173158 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.173134 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.173249 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.173313 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.173474 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.173514 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.173534 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.173548 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.173842 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7555c" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.174203 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-g7g8p\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.174209 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nhhw5\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.174219 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fbpzc\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.174256 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.174271 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.174369 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.174403 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-z6zr8\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.174535 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.174724 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.174810 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.174822 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dx27l\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.176005 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.177426 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.177668 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.177799 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.178206 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dzkmk\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.178461 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.178650 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-phxbx\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.178774 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.178938 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.179399 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.179607 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.179803 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.186319 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:07.187145 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.186487 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.190992 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.191054 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.191628 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.192107 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.192202 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-f84z5\"" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.209549 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal"] Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.209856 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.209928 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.220127 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 17:11:06 +0000 UTC" deadline="2028-04-21 08:36:09.9971782 +0000 UTC" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.220167 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15663h20m2.777013887s" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.220315 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal"] Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.223251 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.242766 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rfjjs" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.259166 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.259631 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rfjjs" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.261313 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.266919 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-run-netns\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.266943 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b996b12-627f-434f-bc7a-6abe5d7907e3-agent-certs\") pod \"konnectivity-agent-v2p4n\" (UID: \"9b996b12-627f-434f-bc7a-6abe5d7907e3\") " pod="kube-system/konnectivity-agent-v2p4n" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.266959 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-sys\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.266997 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-var-lib-cni-bin\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267013 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-var-lib-kubelet\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267035 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-run-multus-certs\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267056 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-env-overrides\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.321617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267079 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-kubernetes\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.322408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267113 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwbhr\" (UniqueName: \"kubernetes.io/projected/a7e40ea6-2b14-45a6-a6da-7b24869031d3-kube-api-access-cwbhr\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.322408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267135 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ed994a80-13df-422c-be49-efca7eebe7d7-iptables-alerter-script\") pod \"iptables-alerter-m8fbb\" (UID: \"ed994a80-13df-422c-be49-efca7eebe7d7\") " pod="openshift-network-operator/iptables-alerter-m8fbb" Jul 08 17:16:07.322408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267152 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-modprobe-d\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.322408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267170 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-lib-modules\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.322408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267194 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9t66\" (UniqueName: \"kubernetes.io/projected/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-kube-api-access-d9t66\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.322408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267216 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-registration-dir\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.322408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267236 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-etc-selinux\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.322408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267262 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-cni-binary-copy\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.322408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267284 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-systemd-units\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.322408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267308 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-slash\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.322408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267325 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.322408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267350 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d3f876c-9309-476f-b7ff-88c6afd11721-host\") pod \"node-ca-7555c\" (UID: \"5d3f876c-9309-476f-b7ff-88c6afd11721\") " pod="openshift-image-registry/node-ca-7555c" Jul 08 17:16:07.322408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267366 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.322408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267382 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-socket-dir\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.322408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267396 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-device-dir\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.322408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267410 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-cni-netd\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.323014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267424 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc2b8\" (UniqueName: \"kubernetes.io/projected/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-kube-api-access-gc2b8\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.323014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267440 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-run-openvswitch\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.323014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267456 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-run-ovn-kubernetes\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.323014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267472 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-cni-bin\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.323014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267485 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-run\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.323014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267498 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bpmq\" (UniqueName: \"kubernetes.io/projected/5d3f876c-9309-476f-b7ff-88c6afd11721-kube-api-access-8bpmq\") pod \"node-ca-7555c\" (UID: \"5d3f876c-9309-476f-b7ff-88c6afd11721\") " pod="openshift-image-registry/node-ca-7555c" Jul 08 17:16:07.323014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267525 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-sysconfig\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.323014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267539 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-system-cni-dir\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.323014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267553 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-var-lib-cni-multus\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.323014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267578 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmwzw\" (UniqueName: \"kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw\") pod \"network-check-target-7n9w9\" (UID: \"b7f1f18b-4d22-40ed-9184-95cc2ae301be\") " pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:07.323014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267610 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-kubelet\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.323014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267630 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-multus-daemon-config\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.323014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267644 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret\") pod \"global-pull-secret-syncer-pqz5t\" (UID: \"17ebcc8b-068a-42c0-b17f-ebae02312e5b\") " pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:07.323014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267657 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-ovnkube-config\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.323014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267690 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpc6j\" (UniqueName: \"kubernetes.io/projected/c3b44d93-d73a-4570-ada2-b13ba1281c59-kube-api-access-kpc6j\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.323014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267717 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m84l9\" (UniqueName: \"kubernetes.io/projected/abdf9cce-f248-42d1-8b24-1f03dcc28ae6-kube-api-access-m84l9\") pod \"node-resolver-xntds\" (UID: \"abdf9cce-f248-42d1-8b24-1f03dcc28ae6\") " pod="openshift-dns/node-resolver-xntds" Jul 08 17:16:07.323684 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267735 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-ovn-node-metrics-cert\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.323684 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267757 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgh7f\" (UniqueName: \"kubernetes.io/projected/ed994a80-13df-422c-be49-efca7eebe7d7-kube-api-access-mgh7f\") pod \"iptables-alerter-m8fbb\" (UID: \"ed994a80-13df-422c-be49-efca7eebe7d7\") " pod="openshift-network-operator/iptables-alerter-m8fbb" Jul 08 17:16:07.323684 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267774 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-tuned\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.323684 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267788 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-multus-socket-dir-parent\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.323684 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267803 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-run-k8s-cni-cncf-io\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.323684 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267816 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/17ebcc8b-068a-42c0-b17f-ebae02312e5b-kubelet-config\") pod \"global-pull-secret-syncer-pqz5t\" (UID: \"17ebcc8b-068a-42c0-b17f-ebae02312e5b\") " pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:07.323684 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267833 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-var-lib-openvswitch\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.323684 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267856 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-ovnkube-script-lib\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.323684 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267878 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b996b12-627f-434f-bc7a-6abe5d7907e3-konnectivity-ca\") pod \"konnectivity-agent-v2p4n\" (UID: \"9b996b12-627f-434f-bc7a-6abe5d7907e3\") " pod="kube-system/konnectivity-agent-v2p4n" Jul 08 17:16:07.323684 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267893 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-host\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.323684 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267914 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3b44d93-d73a-4570-ada2-b13ba1281c59-tmp\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.323684 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267934 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/abdf9cce-f248-42d1-8b24-1f03dcc28ae6-hosts-file\") pod \"node-resolver-xntds\" (UID: \"abdf9cce-f248-42d1-8b24-1f03dcc28ae6\") " pod="openshift-dns/node-resolver-xntds" Jul 08 17:16:07.323684 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267948 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-system-cni-dir\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.323684 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267963 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-multus-conf-dir\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.323684 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.267984 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-systemd\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.323684 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-cnibin\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.324235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268018 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.324235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268034 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.324235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268057 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-cnibin\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.324235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268094 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-run-netns\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.324235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268111 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs\") pod \"network-metrics-daemon-8dk7z\" (UID: \"78604c5e-b53b-4761-b62e-b7f964af3e44\") " pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:07.324235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268126 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8g85\" (UniqueName: \"kubernetes.io/projected/78604c5e-b53b-4761-b62e-b7f964af3e44-kube-api-access-n8g85\") pod \"network-metrics-daemon-8dk7z\" (UID: \"78604c5e-b53b-4761-b62e-b7f964af3e44\") " pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:07.324235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268146 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abdf9cce-f248-42d1-8b24-1f03dcc28ae6-tmp-dir\") pod \"node-resolver-xntds\" (UID: \"abdf9cce-f248-42d1-8b24-1f03dcc28ae6\") " pod="openshift-dns/node-resolver-xntds" Jul 08 17:16:07.324235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-os-release\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.324235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268184 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-etc-kubernetes\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.324235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268203 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-run-systemd\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.324235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268233 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d3f876c-9309-476f-b7ff-88c6afd11721-serviceca\") pod \"node-ca-7555c\" (UID: \"5d3f876c-9309-476f-b7ff-88c6afd11721\") " pod="openshift-image-registry/node-ca-7555c" Jul 08 17:16:07.324235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268250 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm9vx\" (UniqueName: \"kubernetes.io/projected/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-kube-api-access-dm9vx\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.324235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268266 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/17ebcc8b-068a-42c0-b17f-ebae02312e5b-dbus\") pod \"global-pull-secret-syncer-pqz5t\" (UID: \"17ebcc8b-068a-42c0-b17f-ebae02312e5b\") " pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:07.324235 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.268228 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:16:07.324235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268287 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed994a80-13df-422c-be49-efca7eebe7d7-host-slash\") pod \"iptables-alerter-m8fbb\" (UID: \"ed994a80-13df-422c-be49-efca7eebe7d7\") " pod="openshift-network-operator/iptables-alerter-m8fbb" Jul 08 17:16:07.324235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268311 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-sysctl-d\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.324235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268326 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-sysctl-conf\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.324829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268333 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed994a80-13df-422c-be49-efca7eebe7d7-host-slash\") pod \"iptables-alerter-m8fbb\" (UID: \"ed994a80-13df-422c-be49-efca7eebe7d7\") " pod="openshift-network-operator/iptables-alerter-m8fbb" Jul 08 17:16:07.324829 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.268348 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs podName:78604c5e-b53b-4761-b62e-b7f964af3e44 nodeName:}" failed. No retries permitted until 2026-07-08 17:16:07.76833027 +0000 UTC m=+1.994312367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs") pod "network-metrics-daemon-8dk7z" (UID: "78604c5e-b53b-4761-b62e-b7f964af3e44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:16:07.324829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268386 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-var-lib-kubelet\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.324829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268402 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.324829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268417 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-etc-openvswitch\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.324829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268438 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-run-ovn\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.324829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268464 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-node-log\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.324829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268482 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-log-socket\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.324829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268502 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-sys-fs\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.324829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268521 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-multus-cni-dir\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.324829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268540 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-hostroot\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.324829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268561 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-os-release\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.324829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.268609 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-cni-binary-copy\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.324829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.270703 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ed994a80-13df-422c-be49-efca7eebe7d7-iptables-alerter-script\") pod \"iptables-alerter-m8fbb\" (UID: \"ed994a80-13df-422c-be49-efca7eebe7d7\") " pod="openshift-network-operator/iptables-alerter-m8fbb" Jul 08 17:16:07.324829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.284722 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 17:16:07.324829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.298599 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8g85\" (UniqueName: \"kubernetes.io/projected/78604c5e-b53b-4761-b62e-b7f964af3e44-kube-api-access-n8g85\") pod \"network-metrics-daemon-8dk7z\" (UID: \"78604c5e-b53b-4761-b62e-b7f964af3e44\") " pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:07.325358 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.298601 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgh7f\" (UniqueName: \"kubernetes.io/projected/ed994a80-13df-422c-be49-efca7eebe7d7-kube-api-access-mgh7f\") pod \"iptables-alerter-m8fbb\" (UID: \"ed994a80-13df-422c-be49-efca7eebe7d7\") " pod="openshift-network-operator/iptables-alerter-m8fbb" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.369448 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-etc-kubernetes\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.369480 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-run-systemd\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.369495 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d3f876c-9309-476f-b7ff-88c6afd11721-serviceca\") pod \"node-ca-7555c\" (UID: \"5d3f876c-9309-476f-b7ff-88c6afd11721\") " pod="openshift-image-registry/node-ca-7555c" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.369516 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm9vx\" (UniqueName: \"kubernetes.io/projected/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-kube-api-access-dm9vx\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.369567 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-etc-kubernetes\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.369572 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/17ebcc8b-068a-42c0-b17f-ebae02312e5b-dbus\") pod \"global-pull-secret-syncer-pqz5t\" (UID: \"17ebcc8b-068a-42c0-b17f-ebae02312e5b\") " pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.369673 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-sysctl-d\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.369567 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-run-systemd\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.369692 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-sysctl-conf\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.369724 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-var-lib-kubelet\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.369746 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.369777 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/17ebcc8b-068a-42c0-b17f-ebae02312e5b-dbus\") pod \"global-pull-secret-syncer-pqz5t\" (UID: \"17ebcc8b-068a-42c0-b17f-ebae02312e5b\") " pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.369830 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-sysctl-conf\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.369769 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-etc-openvswitch\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370065 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-run-ovn\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370115 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-node-log\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370145 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-log-socket\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.451863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370177 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-sys-fs\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370191 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-var-lib-kubelet\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370206 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-multus-cni-dir\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370229 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5d3f876c-9309-476f-b7ff-88c6afd11721-serviceca\") pod \"node-ca-7555c\" (UID: \"5d3f876c-9309-476f-b7ff-88c6afd11721\") " pod="openshift-image-registry/node-ca-7555c" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370252 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370288 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-node-log\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370236 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-hostroot\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370318 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-etc-openvswitch\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370306 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-hostroot\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370347 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-os-release\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370342 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-log-socket\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370407 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-run-ovn\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370423 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-cni-binary-copy\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370468 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-run-netns\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370487 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-multus-cni-dir\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370498 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b996b12-627f-434f-bc7a-6abe5d7907e3-agent-certs\") pod \"konnectivity-agent-v2p4n\" (UID: \"9b996b12-627f-434f-bc7a-6abe5d7907e3\") " pod="kube-system/konnectivity-agent-v2p4n" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370541 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-run-netns\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370550 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-sys\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.453517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370600 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-os-release\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.454636 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370617 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-sysctl-d\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.454636 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370620 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-var-lib-cni-bin\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.454636 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370679 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-sys-fs\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.454636 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370687 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-sys\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.454636 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370687 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-var-lib-kubelet\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.454636 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370732 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-run-multus-certs\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.454636 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370739 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-var-lib-cni-bin\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.454636 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-env-overrides\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.454636 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370760 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-var-lib-kubelet\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.454636 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370844 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-run-multus-certs\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.454636 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370963 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-kubernetes\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.454636 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.370993 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwbhr\" (UniqueName: \"kubernetes.io/projected/a7e40ea6-2b14-45a6-a6da-7b24869031d3-kube-api-access-cwbhr\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.454636 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371052 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-modprobe-d\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.454636 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371081 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-lib-modules\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.454636 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371131 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9t66\" (UniqueName: \"kubernetes.io/projected/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-kube-api-access-d9t66\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.454636 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371180 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-registration-dir\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.454636 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371182 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-cni-binary-copy\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.455900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371204 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-etc-selinux\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.455900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371244 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-env-overrides\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.455900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371246 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-cni-binary-copy\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.455900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371279 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-kubernetes\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.455900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371321 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-systemd-units\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.455900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371361 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-slash\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.455900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371388 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.455900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371426 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d3f876c-9309-476f-b7ff-88c6afd11721-host\") pod \"node-ca-7555c\" (UID: \"5d3f876c-9309-476f-b7ff-88c6afd11721\") " pod="openshift-image-registry/node-ca-7555c" Jul 08 17:16:07.455900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371459 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.455900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371476 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-slash\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.455900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371492 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-socket-dir\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.455900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371523 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-device-dir\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.455900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371532 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-systemd-units\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.455900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371561 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-cni-netd\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.455900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371642 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-registration-dir\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.455900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371676 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc2b8\" (UniqueName: \"kubernetes.io/projected/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-kube-api-access-gc2b8\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.455900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371699 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-etc-selinux\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.460400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371754 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-run-openvswitch\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.460400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371776 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-lib-modules\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.460400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371813 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-device-dir\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.460400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371703 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-run-openvswitch\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.460400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371862 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a7e40ea6-2b14-45a6-a6da-7b24869031d3-socket-dir\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.460400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371874 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-run-ovn-kubernetes\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.460400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371904 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-cni-bin\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.460400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371937 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-run\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.460400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371967 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bpmq\" (UniqueName: \"kubernetes.io/projected/5d3f876c-9309-476f-b7ff-88c6afd11721-kube-api-access-8bpmq\") pod \"node-ca-7555c\" (UID: \"5d3f876c-9309-476f-b7ff-88c6afd11721\") " pod="openshift-image-registry/node-ca-7555c" Jul 08 17:16:07.460400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.371981 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.460400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372000 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-sysconfig\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.460400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372029 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-system-cni-dir\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.460400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372055 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5d3f876c-9309-476f-b7ff-88c6afd11721-host\") pod \"node-ca-7555c\" (UID: \"5d3f876c-9309-476f-b7ff-88c6afd11721\") " pod="openshift-image-registry/node-ca-7555c" Jul 08 17:16:07.460400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372067 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-var-lib-cni-multus\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.460400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372100 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmwzw\" (UniqueName: \"kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw\") pod \"network-check-target-7n9w9\" (UID: \"b7f1f18b-4d22-40ed-9184-95cc2ae301be\") " pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:07.460400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372148 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-cni-netd\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.460400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372193 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-kubelet\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.461278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372229 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-multus-daemon-config\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.461278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372259 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret\") pod \"global-pull-secret-syncer-pqz5t\" (UID: \"17ebcc8b-068a-42c0-b17f-ebae02312e5b\") " pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:07.461278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372294 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-ovnkube-config\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.461278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372304 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-run-ovn-kubernetes\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.461278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372328 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpc6j\" (UniqueName: \"kubernetes.io/projected/c3b44d93-d73a-4570-ada2-b13ba1281c59-kube-api-access-kpc6j\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.461278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372347 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-cni-bin\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.461278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372360 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m84l9\" (UniqueName: \"kubernetes.io/projected/abdf9cce-f248-42d1-8b24-1f03dcc28ae6-kube-api-access-m84l9\") pod \"node-resolver-xntds\" (UID: \"abdf9cce-f248-42d1-8b24-1f03dcc28ae6\") " pod="openshift-dns/node-resolver-xntds" Jul 08 17:16:07.461278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372397 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-ovn-node-metrics-cert\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.461278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372404 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-run\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.461278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372398 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-cni-binary-copy\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.461278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372422 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-sysconfig\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.461278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372465 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-tuned\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.461278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372498 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-multus-socket-dir-parent\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.461278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372521 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-run-k8s-cni-cncf-io\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.461278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372552 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/17ebcc8b-068a-42c0-b17f-ebae02312e5b-kubelet-config\") pod \"global-pull-secret-syncer-pqz5t\" (UID: \"17ebcc8b-068a-42c0-b17f-ebae02312e5b\") " pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:07.461278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372576 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-var-lib-cni-multus\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.461278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372770 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.462153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372600 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-var-lib-openvswitch\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.462153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372854 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/17ebcc8b-068a-42c0-b17f-ebae02312e5b-kubelet-config\") pod \"global-pull-secret-syncer-pqz5t\" (UID: \"17ebcc8b-068a-42c0-b17f-ebae02312e5b\") " pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:07.462153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372891 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-ovnkube-script-lib\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.462153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372929 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-modprobe-d\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.462153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.372973 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-ovnkube-config\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.462153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373003 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b996b12-627f-434f-bc7a-6abe5d7907e3-konnectivity-ca\") pod \"konnectivity-agent-v2p4n\" (UID: \"9b996b12-627f-434f-bc7a-6abe5d7907e3\") " pod="kube-system/konnectivity-agent-v2p4n" Jul 08 17:16:07.462153 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.373042 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:16:07.462153 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.373115 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret podName:17ebcc8b-068a-42c0-b17f-ebae02312e5b nodeName:}" failed. No retries permitted until 2026-07-08 17:16:07.873093217 +0000 UTC m=+2.099075322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret") pod "global-pull-secret-syncer-pqz5t" (UID: "17ebcc8b-068a-42c0-b17f-ebae02312e5b") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:16:07.462153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373176 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-system-cni-dir\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.462153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373214 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-host-kubelet\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.462153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-host\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.462153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373388 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3b44d93-d73a-4570-ada2-b13ba1281c59-tmp\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.462153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373420 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/abdf9cce-f248-42d1-8b24-1f03dcc28ae6-hosts-file\") pod \"node-resolver-xntds\" (UID: \"abdf9cce-f248-42d1-8b24-1f03dcc28ae6\") " pod="openshift-dns/node-resolver-xntds" Jul 08 17:16:07.462153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373507 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-system-cni-dir\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.462153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373508 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-run-k8s-cni-cncf-io\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.462153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373541 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-multus-conf-dir\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.462153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373606 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b996b12-627f-434f-bc7a-6abe5d7907e3-agent-certs\") pod \"konnectivity-agent-v2p4n\" (UID: \"9b996b12-627f-434f-bc7a-6abe5d7907e3\") " pod="kube-system/konnectivity-agent-v2p4n" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373621 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-systemd\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373648 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-cnibin\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373679 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373688 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-host\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373745 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b996b12-627f-434f-bc7a-6abe5d7907e3-konnectivity-ca\") pod \"konnectivity-agent-v2p4n\" (UID: \"9b996b12-627f-434f-bc7a-6abe5d7907e3\") " pod="kube-system/konnectivity-agent-v2p4n" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373781 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-cnibin\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373839 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-var-lib-openvswitch\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373850 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373895 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/abdf9cce-f248-42d1-8b24-1f03dcc28ae6-hosts-file\") pod \"node-resolver-xntds\" (UID: \"abdf9cce-f248-42d1-8b24-1f03dcc28ae6\") " pod="openshift-dns/node-resolver-xntds" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373922 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-system-cni-dir\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373941 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-systemd\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.373978 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-cnibin\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.374028 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.374053 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-run-netns\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.374034 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-multus-conf-dir\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.374099 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-ovnkube-script-lib\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.374150 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-host-run-netns\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.463891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.374230 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abdf9cce-f248-42d1-8b24-1f03dcc28ae6-tmp-dir\") pod \"node-resolver-xntds\" (UID: \"abdf9cce-f248-42d1-8b24-1f03dcc28ae6\") " pod="openshift-dns/node-resolver-xntds" Jul 08 17:16:07.464764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.374263 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-os-release\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.464764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.374344 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-multus-socket-dir-parent\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.464764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.374435 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-os-release\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.464764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.374479 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-cnibin\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.464764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.374513 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.464764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.374638 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-multus-daemon-config\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.464764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.374785 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abdf9cce-f248-42d1-8b24-1f03dcc28ae6-tmp-dir\") pod \"node-resolver-xntds\" (UID: \"abdf9cce-f248-42d1-8b24-1f03dcc28ae6\") " pod="openshift-dns/node-resolver-xntds" Jul 08 17:16:07.464764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.376189 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3b44d93-d73a-4570-ada2-b13ba1281c59-tmp\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.464764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.377788 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-ovn-node-metrics-cert\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.464764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.382517 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c3b44d93-d73a-4570-ada2-b13ba1281c59-etc-tuned\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.464764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.405020 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm9vx\" (UniqueName: \"kubernetes.io/projected/9adf63f6-5d10-4065-aa17-9fd6fdb55f1b-kube-api-access-dm9vx\") pod \"multus-9wmcv\" (UID: \"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b\") " pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.464764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.405327 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc2b8\" (UniqueName: \"kubernetes.io/projected/d6f8a5dc-42df-4123-b77e-550d7f44c9bc-kube-api-access-gc2b8\") pod \"ovnkube-node-jk4p4\" (UID: \"d6f8a5dc-42df-4123-b77e-550d7f44c9bc\") " pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.464764 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.441297 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:16:07.464764 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.441328 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:16:07.464764 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.441344 2578 projected.go:196] Error preparing data for projected volume kube-api-access-mmwzw for pod openshift-network-diagnostics/network-check-target-7n9w9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:16:07.464764 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.441410 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw podName:b7f1f18b-4d22-40ed-9184-95cc2ae301be nodeName:}" failed. No retries permitted until 2026-07-08 17:16:07.941395575 +0000 UTC m=+2.167377689 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mmwzw" (UniqueName: "kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw") pod "network-check-target-7n9w9" (UID: "b7f1f18b-4d22-40ed-9184-95cc2ae301be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:16:07.464764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.445868 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9t66\" (UniqueName: \"kubernetes.io/projected/f238c743-a6a4-4b86-b9ef-081f6b2ae9d1-kube-api-access-d9t66\") pod \"multus-additional-cni-plugins-2hjbv\" (UID: \"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1\") " pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.465851 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.450856 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwbhr\" (UniqueName: \"kubernetes.io/projected/a7e40ea6-2b14-45a6-a6da-7b24869031d3-kube-api-access-cwbhr\") pod \"aws-ebs-csi-driver-node-4fqbt\" (UID: \"a7e40ea6-2b14-45a6-a6da-7b24869031d3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.465851 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.450857 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bpmq\" (UniqueName: \"kubernetes.io/projected/5d3f876c-9309-476f-b7ff-88c6afd11721-kube-api-access-8bpmq\") pod \"node-ca-7555c\" (UID: \"5d3f876c-9309-476f-b7ff-88c6afd11721\") " pod="openshift-image-registry/node-ca-7555c" Jul 08 17:16:07.465851 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.451169 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m84l9\" (UniqueName: \"kubernetes.io/projected/abdf9cce-f248-42d1-8b24-1f03dcc28ae6-kube-api-access-m84l9\") pod \"node-resolver-xntds\" (UID: \"abdf9cce-f248-42d1-8b24-1f03dcc28ae6\") " pod="openshift-dns/node-resolver-xntds" Jul 08 17:16:07.465851 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.452407 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpc6j\" (UniqueName: \"kubernetes.io/projected/c3b44d93-d73a-4570-ada2-b13ba1281c59-kube-api-access-kpc6j\") pod \"tuned-586wx\" (UID: \"c3b44d93-d73a-4570-ada2-b13ba1281c59\") " pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.598887 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.511825 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m8fbb" Jul 08 17:16:07.598887 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.531725 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:07.598887 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.561059 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v2p4n" Jul 08 17:16:07.598887 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:07.565964 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod409dd90d59cc55e7490a78f8e6f69617.slice/crio-cd9cb28616684bf57fe91647cbaaf7dc526a3b4e07068ca449186d3d13db01ad WatchSource:0}: Error finding container cd9cb28616684bf57fe91647cbaaf7dc526a3b4e07068ca449186d3d13db01ad: Status 404 returned error can't find the container with id cd9cb28616684bf57fe91647cbaaf7dc526a3b4e07068ca449186d3d13db01ad Jul 08 17:16:07.598887 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:07.566432 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded994a80_13df_422c_be49_efca7eebe7d7.slice/crio-a29524fb2e62833e17452066c235c812fad63b8b5f06cc2110fe819e5eb87a32 WatchSource:0}: Error finding container a29524fb2e62833e17452066c235c812fad63b8b5f06cc2110fe819e5eb87a32: Status 404 returned error can't find the container with id a29524fb2e62833e17452066c235c812fad63b8b5f06cc2110fe819e5eb87a32 Jul 08 17:16:07.598887 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:07.567339 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6f8a5dc_42df_4123_b77e_550d7f44c9bc.slice/crio-bc5eca63de2dacea1f04f2277fa9ff651def7f78802ff84fbe4687718e2371d5 WatchSource:0}: Error finding container bc5eca63de2dacea1f04f2277fa9ff651def7f78802ff84fbe4687718e2371d5: Status 404 returned error can't find the container with id bc5eca63de2dacea1f04f2277fa9ff651def7f78802ff84fbe4687718e2371d5 Jul 08 17:16:07.598887 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:07.572073 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04a30fea7d8cb69c63b01d701b00e3be.slice/crio-c26eb125e71e70bf090fed5f3b67437ae8e893e4acea2bedc91226e939e30d4e WatchSource:0}: Error finding container c26eb125e71e70bf090fed5f3b67437ae8e893e4acea2bedc91226e939e30d4e: Status 404 returned error can't find the container with id c26eb125e71e70bf090fed5f3b67437ae8e893e4acea2bedc91226e939e30d4e Jul 08 17:16:07.598887 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.572496 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 17:16:07.598887 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:07.574100 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b996b12_627f_434f_bc7a_6abe5d7907e3.slice/crio-034f7eaa592e8e41cfe4e26b5272d79d9288eee4da1fa35e815f333f2df64b6b WatchSource:0}: Error finding container 034f7eaa592e8e41cfe4e26b5272d79d9288eee4da1fa35e815f333f2df64b6b: Status 404 returned error can't find the container with id 034f7eaa592e8e41cfe4e26b5272d79d9288eee4da1fa35e815f333f2df64b6b Jul 08 17:16:07.598887 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.585574 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" Jul 08 17:16:07.598887 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:07.595604 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e40ea6_2b14_45a6_a6da_7b24869031d3.slice/crio-ccb2bdee726bc99591d943a77f990047fee6e58c789c8d8a6bc1b6250fca25f3 WatchSource:0}: Error finding container ccb2bdee726bc99591d943a77f990047fee6e58c789c8d8a6bc1b6250fca25f3: Status 404 returned error can't find the container with id ccb2bdee726bc99591d943a77f990047fee6e58c789c8d8a6bc1b6250fca25f3 Jul 08 17:16:07.762554 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.613010 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-586wx" Jul 08 17:16:07.762554 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.618398 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xntds" Jul 08 17:16:07.762554 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:07.621474 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3b44d93_d73a_4570_ada2_b13ba1281c59.slice/crio-a01aef98cfd150923e36f341ecedb000fbd60d4300a4c5166a8a9778a436db24 WatchSource:0}: Error finding container a01aef98cfd150923e36f341ecedb000fbd60d4300a4c5166a8a9778a436db24: Status 404 returned error can't find the container with id a01aef98cfd150923e36f341ecedb000fbd60d4300a4c5166a8a9778a436db24 Jul 08 17:16:07.762554 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.626969 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7555c" Jul 08 17:16:07.762554 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:07.627254 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabdf9cce_f248_42d1_8b24_1f03dcc28ae6.slice/crio-ac9ff2a0adbf99258bd44ef81bd4b4ba5f4433405398b7baa1bdbd8da06c7559 WatchSource:0}: Error finding container ac9ff2a0adbf99258bd44ef81bd4b4ba5f4433405398b7baa1bdbd8da06c7559: Status 404 returned error can't find the container with id ac9ff2a0adbf99258bd44ef81bd4b4ba5f4433405398b7baa1bdbd8da06c7559 Jul 08 17:16:07.762554 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:07.636053 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d3f876c_9309_476f_b7ff_88c6afd11721.slice/crio-6e82f12682ce734473f784b661410cc2a03224a56cad21933a40ab1452e8fa25 WatchSource:0}: Error finding container 6e82f12682ce734473f784b661410cc2a03224a56cad21933a40ab1452e8fa25: Status 404 returned error can't find the container with id 6e82f12682ce734473f784b661410cc2a03224a56cad21933a40ab1452e8fa25 Jul 08 17:16:07.762554 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.669216 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9wmcv" Jul 08 17:16:07.762554 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.686562 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2hjbv" Jul 08 17:16:07.762554 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:07.694804 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf238c743_a6a4_4b86_b9ef_081f6b2ae9d1.slice/crio-175c18ab7313b9fcced01a83e1e7d36ab47094b1c7a25ed4c9e71269e1781f69 WatchSource:0}: Error finding container 175c18ab7313b9fcced01a83e1e7d36ab47094b1c7a25ed4c9e71269e1781f69: Status 404 returned error can't find the container with id 175c18ab7313b9fcced01a83e1e7d36ab47094b1c7a25ed4c9e71269e1781f69 Jul 08 17:16:07.762554 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.696574 2578 kuberuntime_manager.go:1459] "Unhandled Error" err="init container egress-router-binary-copy start failed in pod multus-additional-cni-plugins-2hjbv_openshift-multus(f238c743-a6a4-4b86-b9ef-081f6b2ae9d1): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 08 17:16:07.762554 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.696624 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"egress-router-binary-copy\" with ErrImagePull: \"pull QPS exceeded\"" pod="openshift-multus/multus-additional-cni-plugins-2hjbv" podUID="f238c743-a6a4-4b86-b9ef-081f6b2ae9d1" Jul 08 17:16:07.777386 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.776175 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs\") pod \"network-metrics-daemon-8dk7z\" (UID: \"78604c5e-b53b-4761-b62e-b7f964af3e44\") " pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:07.777386 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.776322 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:16:07.777386 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.776418 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs podName:78604c5e-b53b-4761-b62e-b7f964af3e44 nodeName:}" failed. No retries permitted until 2026-07-08 17:16:08.776400023 +0000 UTC m=+3.002382122 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs") pod "network-metrics-daemon-8dk7z" (UID: "78604c5e-b53b-4761-b62e-b7f964af3e44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:16:07.908994 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.876799 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret\") pod \"global-pull-secret-syncer-pqz5t\" (UID: \"17ebcc8b-068a-42c0-b17f-ebae02312e5b\") " pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:07.908994 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.876925 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:16:07.908994 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.876977 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret podName:17ebcc8b-068a-42c0-b17f-ebae02312e5b nodeName:}" failed. No retries permitted until 2026-07-08 17:16:08.876963922 +0000 UTC m=+3.102946020 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret") pod "global-pull-secret-syncer-pqz5t" (UID: "17ebcc8b-068a-42c0-b17f-ebae02312e5b") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:16:08.096055 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.994439 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmwzw\" (UniqueName: \"kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw\") pod \"network-check-target-7n9w9\" (UID: \"b7f1f18b-4d22-40ed-9184-95cc2ae301be\") " pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:08.096055 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.994638 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:16:08.096055 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.994670 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:16:08.096055 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.994685 2578 projected.go:196] Error preparing data for projected volume kube-api-access-mmwzw for pod openshift-network-diagnostics/network-check-target-7n9w9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:16:08.096055 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:07.994749 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw podName:b7f1f18b-4d22-40ed-9184-95cc2ae301be nodeName:}" failed. No retries permitted until 2026-07-08 17:16:08.994728453 +0000 UTC m=+3.220710554 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mmwzw" (UniqueName: "kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw") pod "network-check-target-7n9w9" (UID: "b7f1f18b-4d22-40ed-9184-95cc2ae301be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:16:08.096055 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:07.996948 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:16:08.319880 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:08.262864 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 17:11:07 +0000 UTC" deadline="2028-02-09 23:50:14.126544847 +0000 UTC" Jul 08 17:16:08.319880 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:08.262924 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13950h34m5.863625058s" Jul 08 17:16:08.390362 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:08.390333 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:08.390526 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:08.390455 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:08.406029 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:08.405988 2578 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b2e954d9267f62174ff45a44976fa7a47e58d87fef14ffbb9fc0f8184e722280" Jul 08 17:16:08.406204 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:08.406115 2578 kuberuntime_manager.go:1459] "Unhandled Error" err="container konnectivity-agent start failed in pod konnectivity-agent-v2p4n_kube-system(9b996b12-627f-434f-bc7a-6abe5d7907e3): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" logger="UnhandledError" Jul 08 17:16:08.406204 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:08.406155 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"konnectivity-agent\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="kube-system/konnectivity-agent-v2p4n" podUID="9b996b12-627f-434f-bc7a-6abe5d7907e3" Jul 08 17:16:08.407729 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:08.407679 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" event={"ID":"409dd90d59cc55e7490a78f8e6f69617","Type":"ContainerStarted","Data":"cd9cb28616684bf57fe91647cbaaf7dc526a3b4e07068ca449186d3d13db01ad"} Jul 08 17:16:08.410537 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:08.410441 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" event={"ID":"d6f8a5dc-42df-4123-b77e-550d7f44c9bc","Type":"ContainerStarted","Data":"bc5eca63de2dacea1f04f2277fa9ff651def7f78802ff84fbe4687718e2371d5"} Jul 08 17:16:08.414991 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:08.414962 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m8fbb" event={"ID":"ed994a80-13df-422c-be49-efca7eebe7d7","Type":"ContainerStarted","Data":"a29524fb2e62833e17452066c235c812fad63b8b5f06cc2110fe819e5eb87a32"} Jul 08 17:16:08.424161 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:08.424128 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2hjbv" event={"ID":"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1","Type":"ContainerStarted","Data":"175c18ab7313b9fcced01a83e1e7d36ab47094b1c7a25ed4c9e71269e1781f69"} Jul 08 17:16:08.433491 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:08.433227 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"egress-router-binary-copy\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:49287a1683e255a8c7d468520a3bd9e589ec8c38863ecc28ce4886ed0cce9828\\\": ErrImagePull: pull QPS exceeded\"" pod="openshift-multus/multus-additional-cni-plugins-2hjbv" podUID="f238c743-a6a4-4b86-b9ef-081f6b2ae9d1" Jul 08 17:16:08.438198 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:08.438127 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9wmcv" event={"ID":"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b","Type":"ContainerStarted","Data":"99441a5870a685d10ed775f8589b068c9ab22be74244832b8e8c0b586ac88369"} Jul 08 17:16:08.440249 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:08.440220 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7555c" event={"ID":"5d3f876c-9309-476f-b7ff-88c6afd11721","Type":"ContainerStarted","Data":"6e82f12682ce734473f784b661410cc2a03224a56cad21933a40ab1452e8fa25"} Jul 08 17:16:08.445047 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:08.444972 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xntds" event={"ID":"abdf9cce-f248-42d1-8b24-1f03dcc28ae6","Type":"ContainerStarted","Data":"ac9ff2a0adbf99258bd44ef81bd4b4ba5f4433405398b7baa1bdbd8da06c7559"} Jul 08 17:16:08.449053 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:08.449025 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-586wx" event={"ID":"c3b44d93-d73a-4570-ada2-b13ba1281c59","Type":"ContainerStarted","Data":"a01aef98cfd150923e36f341ecedb000fbd60d4300a4c5166a8a9778a436db24"} Jul 08 17:16:08.459781 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:08.459682 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" event={"ID":"a7e40ea6-2b14-45a6-a6da-7b24869031d3","Type":"ContainerStarted","Data":"ccb2bdee726bc99591d943a77f990047fee6e58c789c8d8a6bc1b6250fca25f3"} Jul 08 17:16:08.468515 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:08.467890 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v2p4n" event={"ID":"9b996b12-627f-434f-bc7a-6abe5d7907e3","Type":"ContainerStarted","Data":"034f7eaa592e8e41cfe4e26b5272d79d9288eee4da1fa35e815f333f2df64b6b"} Jul 08 17:16:08.470681 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:08.470194 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"konnectivity-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b2e954d9267f62174ff45a44976fa7a47e58d87fef14ffbb9fc0f8184e722280\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="kube-system/konnectivity-agent-v2p4n" podUID="9b996b12-627f-434f-bc7a-6abe5d7907e3" Jul 08 17:16:08.471348 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:08.471277 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" event={"ID":"04a30fea7d8cb69c63b01d701b00e3be","Type":"ContainerStarted","Data":"c26eb125e71e70bf090fed5f3b67437ae8e893e4acea2bedc91226e939e30d4e"} Jul 08 17:16:08.801243 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:08.800520 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs\") pod \"network-metrics-daemon-8dk7z\" (UID: \"78604c5e-b53b-4761-b62e-b7f964af3e44\") " pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:08.801243 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:08.800720 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:16:08.801243 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:08.800819 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs podName:78604c5e-b53b-4761-b62e-b7f964af3e44 nodeName:}" failed. No retries permitted until 2026-07-08 17:16:10.800799695 +0000 UTC m=+5.026781800 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs") pod "network-metrics-daemon-8dk7z" (UID: "78604c5e-b53b-4761-b62e-b7f964af3e44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:16:08.901570 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:08.901004 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret\") pod \"global-pull-secret-syncer-pqz5t\" (UID: \"17ebcc8b-068a-42c0-b17f-ebae02312e5b\") " pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:08.901570 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:08.901158 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:16:08.901570 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:08.901223 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret podName:17ebcc8b-068a-42c0-b17f-ebae02312e5b nodeName:}" failed. No retries permitted until 2026-07-08 17:16:10.901203605 +0000 UTC m=+5.127185708 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret") pod "global-pull-secret-syncer-pqz5t" (UID: "17ebcc8b-068a-42c0-b17f-ebae02312e5b") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:16:09.002316 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:09.002278 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmwzw\" (UniqueName: \"kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw\") pod \"network-check-target-7n9w9\" (UID: \"b7f1f18b-4d22-40ed-9184-95cc2ae301be\") " pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:09.002545 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:09.002493 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:16:09.002545 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:09.002517 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:16:09.002545 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:09.002530 2578 projected.go:196] Error preparing data for projected volume kube-api-access-mmwzw for pod openshift-network-diagnostics/network-check-target-7n9w9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:16:09.002749 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:09.002605 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw podName:b7f1f18b-4d22-40ed-9184-95cc2ae301be nodeName:}" failed. No retries permitted until 2026-07-08 17:16:11.002571066 +0000 UTC m=+5.228553183 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mmwzw" (UniqueName: "kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw") pod "network-check-target-7n9w9" (UID: "b7f1f18b-4d22-40ed-9184-95cc2ae301be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:16:09.263633 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:09.263527 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 17:11:07 +0000 UTC" deadline="2028-01-25 19:39:49.526511101 +0000 UTC" Jul 08 17:16:09.263633 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:09.263571 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13586h23m40.26294541s" Jul 08 17:16:09.387899 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:09.387866 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:09.388254 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:09.387902 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:09.388254 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:09.388010 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:09.388254 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:09.388141 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:09.849284 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:09.849060 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:16:10.392157 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:10.389744 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:10.392157 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:10.389879 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:10.818216 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:10.818124 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs\") pod \"network-metrics-daemon-8dk7z\" (UID: \"78604c5e-b53b-4761-b62e-b7f964af3e44\") " pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:10.818389 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:10.818286 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:16:10.818389 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:10.818364 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs podName:78604c5e-b53b-4761-b62e-b7f964af3e44 nodeName:}" failed. No retries permitted until 2026-07-08 17:16:14.818341384 +0000 UTC m=+9.044323496 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs") pod "network-metrics-daemon-8dk7z" (UID: "78604c5e-b53b-4761-b62e-b7f964af3e44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:16:10.918659 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:10.918512 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret\") pod \"global-pull-secret-syncer-pqz5t\" (UID: \"17ebcc8b-068a-42c0-b17f-ebae02312e5b\") " pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:10.918858 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:10.918694 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:16:10.918858 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:10.918765 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret podName:17ebcc8b-068a-42c0-b17f-ebae02312e5b nodeName:}" failed. No retries permitted until 2026-07-08 17:16:14.918746463 +0000 UTC m=+9.144728574 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret") pod "global-pull-secret-syncer-pqz5t" (UID: "17ebcc8b-068a-42c0-b17f-ebae02312e5b") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:16:11.019601 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:11.019552 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmwzw\" (UniqueName: \"kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw\") pod \"network-check-target-7n9w9\" (UID: \"b7f1f18b-4d22-40ed-9184-95cc2ae301be\") " pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:11.019774 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:11.019734 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:16:11.019774 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:11.019754 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:16:11.019774 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:11.019766 2578 projected.go:196] Error preparing data for projected volume kube-api-access-mmwzw for pod openshift-network-diagnostics/network-check-target-7n9w9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:16:11.019943 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:11.019822 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw podName:b7f1f18b-4d22-40ed-9184-95cc2ae301be nodeName:}" failed. No retries permitted until 2026-07-08 17:16:15.019805153 +0000 UTC m=+9.245787267 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mmwzw" (UniqueName: "kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw") pod "network-check-target-7n9w9" (UID: "b7f1f18b-4d22-40ed-9184-95cc2ae301be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:16:11.388172 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:11.388136 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:11.388354 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:11.388192 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:11.388354 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:11.388281 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:11.388464 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:11.388416 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:12.388757 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:12.388724 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:12.389209 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:12.388865 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:13.387927 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:13.387889 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:13.388120 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:13.388038 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:13.388402 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:13.388377 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:13.388520 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:13.388493 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:14.388922 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:14.388891 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:14.389411 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:14.389032 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:14.854247 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:14.854203 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs\") pod \"network-metrics-daemon-8dk7z\" (UID: \"78604c5e-b53b-4761-b62e-b7f964af3e44\") " pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:14.854446 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:14.854424 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:16:14.854525 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:14.854501 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs podName:78604c5e-b53b-4761-b62e-b7f964af3e44 nodeName:}" failed. No retries permitted until 2026-07-08 17:16:22.85448091 +0000 UTC m=+17.080463013 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs") pod "network-metrics-daemon-8dk7z" (UID: "78604c5e-b53b-4761-b62e-b7f964af3e44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:16:14.954764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:14.954669 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret\") pod \"global-pull-secret-syncer-pqz5t\" (UID: \"17ebcc8b-068a-42c0-b17f-ebae02312e5b\") " pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:14.954948 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:14.954875 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:16:14.954948 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:14.954943 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret podName:17ebcc8b-068a-42c0-b17f-ebae02312e5b nodeName:}" failed. No retries permitted until 2026-07-08 17:16:22.954926344 +0000 UTC m=+17.180908441 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret") pod "global-pull-secret-syncer-pqz5t" (UID: "17ebcc8b-068a-42c0-b17f-ebae02312e5b") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:16:15.055445 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:15.055411 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmwzw\" (UniqueName: \"kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw\") pod \"network-check-target-7n9w9\" (UID: \"b7f1f18b-4d22-40ed-9184-95cc2ae301be\") " pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:15.055603 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:15.055572 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:16:15.055657 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:15.055609 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:16:15.055657 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:15.055620 2578 projected.go:196] Error preparing data for projected volume kube-api-access-mmwzw for pod openshift-network-diagnostics/network-check-target-7n9w9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:16:15.055747 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:15.055671 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw podName:b7f1f18b-4d22-40ed-9184-95cc2ae301be nodeName:}" failed. No retries permitted until 2026-07-08 17:16:23.05565337 +0000 UTC m=+17.281635467 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mmwzw" (UniqueName: "kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw") pod "network-check-target-7n9w9" (UID: "b7f1f18b-4d22-40ed-9184-95cc2ae301be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:16:15.388720 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:15.388627 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:15.388881 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:15.388627 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:15.388881 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:15.388779 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:15.388881 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:15.388855 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:16.390278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:16.389764 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:16.390278 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:16.389877 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:17.388197 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:17.388165 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:17.388314 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:17.388212 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:17.388375 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:17.388316 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:17.388465 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:17.388441 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:18.388781 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:18.388749 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:18.389253 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:18.388878 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:19.388289 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:19.388258 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:19.388468 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:19.388258 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:19.388468 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:19.388389 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:19.388607 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:19.388461 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:20.388834 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:20.388801 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:20.389254 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:20.388933 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:21.388702 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:21.388479 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:21.388876 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:21.388480 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:21.388876 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:21.388813 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:21.389288 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:21.388879 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:22.388479 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:22.388442 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:22.388682 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:22.388569 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:22.919856 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:22.919822 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs\") pod \"network-metrics-daemon-8dk7z\" (UID: \"78604c5e-b53b-4761-b62e-b7f964af3e44\") " pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:22.920322 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:22.919997 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:16:22.920322 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:22.920080 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs podName:78604c5e-b53b-4761-b62e-b7f964af3e44 nodeName:}" failed. No retries permitted until 2026-07-08 17:16:38.920058179 +0000 UTC m=+33.146040321 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs") pod "network-metrics-daemon-8dk7z" (UID: "78604c5e-b53b-4761-b62e-b7f964af3e44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:16:23.020694 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:23.020663 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret\") pod \"global-pull-secret-syncer-pqz5t\" (UID: \"17ebcc8b-068a-42c0-b17f-ebae02312e5b\") " pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:23.020851 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:23.020819 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:16:23.020897 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:23.020881 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret podName:17ebcc8b-068a-42c0-b17f-ebae02312e5b nodeName:}" failed. No retries permitted until 2026-07-08 17:16:39.020866999 +0000 UTC m=+33.246849097 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret") pod "global-pull-secret-syncer-pqz5t" (UID: "17ebcc8b-068a-42c0-b17f-ebae02312e5b") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:16:23.121036 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:23.120981 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmwzw\" (UniqueName: \"kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw\") pod \"network-check-target-7n9w9\" (UID: \"b7f1f18b-4d22-40ed-9184-95cc2ae301be\") " pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:23.121211 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:23.121122 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:16:23.121211 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:23.121146 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:16:23.121211 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:23.121158 2578 projected.go:196] Error preparing data for projected volume kube-api-access-mmwzw for pod openshift-network-diagnostics/network-check-target-7n9w9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:16:23.121381 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:23.121215 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw podName:b7f1f18b-4d22-40ed-9184-95cc2ae301be nodeName:}" failed. No retries permitted until 2026-07-08 17:16:39.121195209 +0000 UTC m=+33.347177313 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mmwzw" (UniqueName: "kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw") pod "network-check-target-7n9w9" (UID: "b7f1f18b-4d22-40ed-9184-95cc2ae301be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:16:23.388364 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:23.388276 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:23.388364 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:23.388276 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:23.388611 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:23.388488 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:23.388677 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:23.388614 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:24.390660 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:24.390632 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:24.391018 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:24.390729 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:25.388551 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:25.388512 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:25.388770 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:25.388513 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:25.388770 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:25.388667 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:25.388770 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:25.388735 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:26.390164 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:26.389698 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:26.390164 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:26.389816 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:27.388640 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.388555 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:27.388780 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:27.388679 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:27.388780 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.388730 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:27.388861 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:27.388825 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:27.520871 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.520839 2578 generic.go:356] "Generic (PLEG): container finished" podID="409dd90d59cc55e7490a78f8e6f69617" containerID="2e04a31fea8d5b16483bdff5b6cb1fc9825ac6c0698582b42d2e295340c3f7a9" exitCode=0 Jul 08 17:16:27.521637 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.520913 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" event={"ID":"409dd90d59cc55e7490a78f8e6f69617","Type":"ContainerDied","Data":"2e04a31fea8d5b16483bdff5b6cb1fc9825ac6c0698582b42d2e295340c3f7a9"} Jul 08 17:16:27.524797 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.524557 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" event={"ID":"d6f8a5dc-42df-4123-b77e-550d7f44c9bc","Type":"ContainerStarted","Data":"6236c5644653a336d2dc7a20576750c75c46bdc475003755d799244db1ff9a52"} Jul 08 17:16:27.524933 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.524864 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" event={"ID":"d6f8a5dc-42df-4123-b77e-550d7f44c9bc","Type":"ContainerStarted","Data":"a8b91bd59f0bdb6922c73546925a3737c2d98f33f6b48da37d03b34f918c92e5"} Jul 08 17:16:27.524933 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.524919 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" event={"ID":"d6f8a5dc-42df-4123-b77e-550d7f44c9bc","Type":"ContainerStarted","Data":"fed66252ed497194ff58c1ad4a9b649c22a1922647f0b38f5d9e05b7a24eb1ae"} Jul 08 17:16:27.525033 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.524968 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" event={"ID":"d6f8a5dc-42df-4123-b77e-550d7f44c9bc","Type":"ContainerStarted","Data":"fd01b7e56f2a2cd0e256a3f8e9e084f5184bdc8ccba3809b2295db94e94ea1d5"} Jul 08 17:16:27.525033 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.525014 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" event={"ID":"d6f8a5dc-42df-4123-b77e-550d7f44c9bc","Type":"ContainerStarted","Data":"c7b073048a1df022480b19af40931d0dc967883c3e5972877808b134011e36ee"} Jul 08 17:16:27.525140 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.525063 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" event={"ID":"d6f8a5dc-42df-4123-b77e-550d7f44c9bc","Type":"ContainerStarted","Data":"36b4490a9d125a713a96a952d795a2dcc7c4ded8fd81415ec85abd99ddc708ce"} Jul 08 17:16:27.526613 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.526571 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9wmcv" event={"ID":"9adf63f6-5d10-4065-aa17-9fd6fdb55f1b","Type":"ContainerStarted","Data":"4de41e9d690f18627c6bf3f218ad7e4224ab0ca7960dae8f830f955a364d3c52"} Jul 08 17:16:27.528231 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.528203 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7555c" event={"ID":"5d3f876c-9309-476f-b7ff-88c6afd11721","Type":"ContainerStarted","Data":"a866bd9707d9886c815ae8b961797853727007147c25150a3abe39d6f12a5c23"} Jul 08 17:16:27.529818 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.529786 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xntds" event={"ID":"abdf9cce-f248-42d1-8b24-1f03dcc28ae6","Type":"ContainerStarted","Data":"8aa5e6d3fc53cd101576a959ccd2364b2d33fa2713bd21e7d4602a3728e71d63"} Jul 08 17:16:27.533822 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.533782 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-586wx" event={"ID":"c3b44d93-d73a-4570-ada2-b13ba1281c59","Type":"ContainerStarted","Data":"7f6731c5f32344a0abe460e6ceeb0933042437c7e3722b804f42df91d1767ce9"} Jul 08 17:16:27.535381 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.535352 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" event={"ID":"a7e40ea6-2b14-45a6-a6da-7b24869031d3","Type":"ContainerStarted","Data":"79ea1d989e6d3d982928457572c8ac4c872c099b16fa68c8b5a6dbac1571567d"} Jul 08 17:16:27.536842 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.536814 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v2p4n" event={"ID":"9b996b12-627f-434f-bc7a-6abe5d7907e3","Type":"ContainerStarted","Data":"f870fad31013c8e7b0b745ec381e1de18b95578968fb3b3d1240034f275b0b6d"} Jul 08 17:16:27.538363 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.538342 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" event={"ID":"04a30fea7d8cb69c63b01d701b00e3be","Type":"ContainerStarted","Data":"985298390a43e1a5332fc9d966545b970e15c249624a6ebe97c2f9f7a9d81adc"} Jul 08 17:16:27.565836 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.565749 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-50.ec2.internal" podStartSLOduration=20.565724863 podStartE2EDuration="20.565724863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:16:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:16:27.565333434 +0000 UTC m=+21.791315555" watchObservedRunningTime="2026-07-08 17:16:27.565724863 +0000 UTC m=+21.791706984" Jul 08 17:16:27.609968 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.609921 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-v2p4n" podStartSLOduration=2.583215422 podStartE2EDuration="21.60990492s" totalImagesPullingTime="19.026689498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-08 17:16:27.587495543 +0000 UTC m=+21.813477705" watchObservedRunningTime="2026-07-08 17:16:27.60990492 +0000 UTC m=+21.835887021" Jul 08 17:16:27.630322 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.630261 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-586wx" podStartSLOduration=2.755584716 podStartE2EDuration="21.630240921s" totalImagesPullingTime="18.874656205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:16:27.61024942 +0000 UTC m=+21.836231560" watchObservedRunningTime="2026-07-08 17:16:27.630240921 +0000 UTC m=+21.856223041" Jul 08 17:16:27.630489 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.630390 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xntds" podStartSLOduration=7.803437221 podStartE2EDuration="21.630383034s" totalImagesPullingTime="13.826945813s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:16:27.63019785 +0000 UTC m=+21.856179971" watchObservedRunningTime="2026-07-08 17:16:27.630383034 +0000 UTC m=+21.856365155" Jul 08 17:16:27.716916 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:27.716786 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7555c" podStartSLOduration=3.611567853 podStartE2EDuration="21.716763989s" totalImagesPullingTime="18.105196136s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:16:27.668776289 +0000 UTC m=+21.894758409" watchObservedRunningTime="2026-07-08 17:16:27.716763989 +0000 UTC m=+21.942746141" Jul 08 17:16:28.388974 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:28.388931 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:28.389126 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:28.389086 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:28.542155 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:28.542122 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" event={"ID":"409dd90d59cc55e7490a78f8e6f69617","Type":"ContainerStarted","Data":"ff28a9285b1d150f41dba13297a7526c6232c289c783d10d00ef63dd3c65a7d9"} Jul 08 17:16:28.543754 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:28.543725 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m8fbb" event={"ID":"ed994a80-13df-422c-be49-efca7eebe7d7","Type":"ContainerStarted","Data":"43674f84b1b6a37eaffd1787e39a9137b7c900eb1f61fad743cf32565ac24edb"} Jul 08 17:16:28.545273 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:28.545249 2578 generic.go:356] "Generic (PLEG): container finished" podID="f238c743-a6a4-4b86-b9ef-081f6b2ae9d1" containerID="0e01dff915e18629154a19e6ac4129de93bcf088783419713a4558615d26aa48" exitCode=0 Jul 08 17:16:28.545380 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:28.545331 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2hjbv" event={"ID":"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1","Type":"ContainerDied","Data":"0e01dff915e18629154a19e6ac4129de93bcf088783419713a4558615d26aa48"} Jul 08 17:16:28.563335 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:28.563241 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9wmcv" podStartSLOduration=3.745402386 podStartE2EDuration="22.563224713s" totalImagesPullingTime="18.817822327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:16:27.717512233 +0000 UTC m=+21.943494399" watchObservedRunningTime="2026-07-08 17:16:28.563224713 +0000 UTC m=+22.789206831" Jul 08 17:16:28.595312 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:28.595245 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-50.ec2.internal" podStartSLOduration=21.595225181 podStartE2EDuration="21.595225181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:16:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:16:28.564903133 +0000 UTC m=+22.790885254" watchObservedRunningTime="2026-07-08 17:16:28.595225181 +0000 UTC m=+22.821207302" Jul 08 17:16:28.624860 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:28.624804 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m8fbb" podStartSLOduration=3.699238277 podStartE2EDuration="22.624784434s" totalImagesPullingTime="18.925546157s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:16:28.624774629 +0000 UTC m=+22.850756748" watchObservedRunningTime="2026-07-08 17:16:28.624784434 +0000 UTC m=+22.850766618" Jul 08 17:16:29.080182 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:29.080156 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 17:16:29.267814 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:29.267650 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T17:16:29.080176046Z","UUID":"b0f1a591-0479-45d4-85da-87c12fb344e1","Handler":null,"Name":"","Endpoint":""} Jul 08 17:16:29.269393 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:29.269363 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 17:16:29.269542 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:29.269411 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 17:16:29.388644 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:29.388614 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:29.388776 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:29.388724 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:29.388776 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:29.388755 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:29.388848 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:29.388835 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:29.550323 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:29.550237 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" event={"ID":"d6f8a5dc-42df-4123-b77e-550d7f44c9bc","Type":"ContainerStarted","Data":"ef99b27f41d108df3b1ca88b25dc32657e6843bb85c6af4831875a5be975ce62"} Jul 08 17:16:29.552045 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:29.552024 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" event={"ID":"a7e40ea6-2b14-45a6-a6da-7b24869031d3","Type":"ContainerStarted","Data":"0997010b2a74a7b847d9acd60198a947176a3fd7223a5b0105c2e03c2527e09b"} Jul 08 17:16:30.388728 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:30.388700 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:30.388888 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:30.388804 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:30.556791 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:30.556694 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" event={"ID":"a7e40ea6-2b14-45a6-a6da-7b24869031d3","Type":"ContainerStarted","Data":"a4cb20fec0a48aef92e6218f420831e2675e5a4f9419fe8e143ae4398f3dec9d"} Jul 08 17:16:30.839751 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:30.839669 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-v2p4n" Jul 08 17:16:31.388938 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:31.388899 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:31.389122 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:31.389041 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:31.389122 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:31.389088 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:31.389241 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:31.389204 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:31.566027 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:31.565436 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" event={"ID":"d6f8a5dc-42df-4123-b77e-550d7f44c9bc","Type":"ContainerStarted","Data":"2a1256d28490fe0fe317f10e89dcd68163cbeca86bb1bb50edc7197289266c30"} Jul 08 17:16:31.566027 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:31.565770 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:31.566027 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:31.565902 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:31.585278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:31.585249 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:31.600924 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:31.600867 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" podStartSLOduration=6.672475914 podStartE2EDuration="25.6008472s" totalImagesPullingTime="18.928371286s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:16:31.600167896 +0000 UTC m=+25.826150042" watchObservedRunningTime="2026-07-08 17:16:31.6008472 +0000 UTC m=+25.826829324" Jul 08 17:16:31.601226 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:31.601204 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4fqbt" podStartSLOduration=3.63225635 podStartE2EDuration="25.601199037s" totalImagesPullingTime="21.968942687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:16:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:16:30.587517787 +0000 UTC m=+24.813499908" watchObservedRunningTime="2026-07-08 17:16:31.601199037 +0000 UTC m=+25.827181157" Jul 08 17:16:32.100787 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:32.100748 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-v2p4n" Jul 08 17:16:32.101490 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:32.101465 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-v2p4n" Jul 08 17:16:32.388425 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:32.388344 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:32.388573 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:32.388478 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:32.567954 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:32.567882 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:32.568730 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:32.568652 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-v2p4n" Jul 08 17:16:32.584117 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:32.584095 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:16:33.388227 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:33.388196 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:33.388227 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:33.388219 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:33.388448 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:33.388315 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:33.388488 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:33.388456 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:33.718473 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:33.718401 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pqz5t"] Jul 08 17:16:33.719026 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:33.718491 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:33.719026 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:33.718575 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:33.722309 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:33.722284 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7n9w9"] Jul 08 17:16:33.722431 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:33.722390 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:33.722492 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:33.722469 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:33.734393 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:33.734361 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8dk7z"] Jul 08 17:16:33.734539 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:33.734474 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:33.734626 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:33.734578 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:35.388360 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:35.388167 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:35.388785 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:35.388214 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:35.388785 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:35.388481 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:35.388785 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:35.388543 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:35.388785 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:35.388212 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:35.388785 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:35.388701 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:36.577563 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:36.577525 2578 generic.go:356] "Generic (PLEG): container finished" podID="f238c743-a6a4-4b86-b9ef-081f6b2ae9d1" containerID="51f057e2479c50dc8c1693818622d5b0aca2c10ef46d69a4e95f7b4b9c52cebe" exitCode=0 Jul 08 17:16:36.578096 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:36.577603 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2hjbv" event={"ID":"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1","Type":"ContainerDied","Data":"51f057e2479c50dc8c1693818622d5b0aca2c10ef46d69a4e95f7b4b9c52cebe"} Jul 08 17:16:37.388549 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:37.388519 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:37.388730 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:37.388519 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:37.388730 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:37.388638 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:37.388730 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:37.388634 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:37.388730 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:37.388718 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:37.388864 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:37.388778 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:38.583015 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:38.582978 2578 generic.go:356] "Generic (PLEG): container finished" podID="f238c743-a6a4-4b86-b9ef-081f6b2ae9d1" containerID="c008ff3506964a8a2546a2d5c458b68076f6fce3f3d8a5c75443360b3d25e93c" exitCode=0 Jul 08 17:16:38.583403 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:38.583027 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2hjbv" event={"ID":"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1","Type":"ContainerDied","Data":"c008ff3506964a8a2546a2d5c458b68076f6fce3f3d8a5c75443360b3d25e93c"} Jul 08 17:16:38.941857 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:38.941819 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs\") pod \"network-metrics-daemon-8dk7z\" (UID: \"78604c5e-b53b-4761-b62e-b7f964af3e44\") " pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:38.942008 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:38.941970 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:16:38.942052 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:38.942046 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs podName:78604c5e-b53b-4761-b62e-b7f964af3e44 nodeName:}" failed. No retries permitted until 2026-07-08 17:17:10.942029642 +0000 UTC m=+65.168011740 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs") pod "network-metrics-daemon-8dk7z" (UID: "78604c5e-b53b-4761-b62e-b7f964af3e44") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:16:39.043111 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.043056 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret\") pod \"global-pull-secret-syncer-pqz5t\" (UID: \"17ebcc8b-068a-42c0-b17f-ebae02312e5b\") " pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:39.043302 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:39.043206 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:16:39.043302 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:39.043271 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret podName:17ebcc8b-068a-42c0-b17f-ebae02312e5b nodeName:}" failed. No retries permitted until 2026-07-08 17:17:11.043255657 +0000 UTC m=+65.269237755 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret") pod "global-pull-secret-syncer-pqz5t" (UID: "17ebcc8b-068a-42c0-b17f-ebae02312e5b") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:16:39.143485 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.143449 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmwzw\" (UniqueName: \"kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw\") pod \"network-check-target-7n9w9\" (UID: \"b7f1f18b-4d22-40ed-9184-95cc2ae301be\") " pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:39.143678 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:39.143655 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:16:39.143769 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:39.143688 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:16:39.143769 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:39.143704 2578 projected.go:196] Error preparing data for projected volume kube-api-access-mmwzw for pod openshift-network-diagnostics/network-check-target-7n9w9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:16:39.143862 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:39.143770 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw podName:b7f1f18b-4d22-40ed-9184-95cc2ae301be nodeName:}" failed. No retries permitted until 2026-07-08 17:17:11.143751515 +0000 UTC m=+65.369733626 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mmwzw" (UniqueName: "kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw") pod "network-check-target-7n9w9" (UID: "b7f1f18b-4d22-40ed-9184-95cc2ae301be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:16:39.387973 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.387871 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:39.388146 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.387873 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:39.388146 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:39.388014 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-pqz5t" podUID="17ebcc8b-068a-42c0-b17f-ebae02312e5b" Jul 08 17:16:39.388146 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:39.388094 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:16:39.388146 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.387883 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:39.388332 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:39.388168 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7n9w9" podUID="b7f1f18b-4d22-40ed-9184-95cc2ae301be" Jul 08 17:16:39.491074 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.491047 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-50.ec2.internal" event="NodeReady" Jul 08 17:16:39.491214 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.491174 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 17:16:39.529802 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.529771 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-njbnd"] Jul 08 17:16:39.547660 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.547632 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-699f54966c-dtkx2"] Jul 08 17:16:39.547799 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.547769 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:16:39.550267 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.550227 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dmvf7\"" Jul 08 17:16:39.550397 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.550373 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 08 17:16:39.551054 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.550900 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 08 17:16:39.563496 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.563473 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-njbnd"] Jul 08 17:16:39.563640 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.563496 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.563640 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.563531 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-699f54966c-dtkx2"] Jul 08 17:16:39.563640 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.563554 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mqrpr"] Jul 08 17:16:39.566253 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.566230 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 17:16:39.566524 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.566504 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 17:16:39.566641 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.566546 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tnxw9\"" Jul 08 17:16:39.566823 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.566804 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 17:16:39.572613 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.572576 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 17:16:39.581387 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.581367 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7lmsl"] Jul 08 17:16:39.581522 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.581500 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mqrpr" Jul 08 17:16:39.585710 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.585192 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 17:16:39.585710 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.585646 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 17:16:39.587124 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.587103 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dpkhh\"" Jul 08 17:16:39.601550 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.601530 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7lmsl"] Jul 08 17:16:39.601671 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.601563 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mqrpr"] Jul 08 17:16:39.601725 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.601677 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7lmsl" Jul 08 17:16:39.604296 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.604280 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 17:16:39.604426 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.604302 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 17:16:39.604502 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.604422 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zsfk5\"" Jul 08 17:16:39.604561 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.604502 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 17:16:39.647523 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.647446 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7crcm\" (UniqueName: \"kubernetes.io/projected/c83a300a-7a84-40ed-85fd-baa1a196f0f3-kube-api-access-7crcm\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:16:39.647523 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.647480 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-certificates\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.647523 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.647499 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:16:39.647523 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.647519 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-ca-trust-extracted\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.647760 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.647540 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5q7sz\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-kube-api-access-5q7sz\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.647760 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.647558 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c83a300a-7a84-40ed-85fd-baa1a196f0f3-tmp-dir\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:16:39.647760 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.647576 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-trusted-ca\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.647760 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.647646 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-installation-pull-secrets\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.647760 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.647664 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/626e8de7-fd53-4204-8f73-9d1d41b32d19-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-njbnd\" (UID: \"626e8de7-fd53-4204-8f73-9d1d41b32d19\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:16:39.647760 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.647692 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-image-registry-private-configuration\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.647760 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.647719 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c83a300a-7a84-40ed-85fd-baa1a196f0f3-config-volume\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:16:39.647760 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.647741 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-njbnd\" (UID: \"626e8de7-fd53-4204-8f73-9d1d41b32d19\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:16:39.647974 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.647771 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.647974 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.647785 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-bound-sa-token\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.749098 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.748880 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7crcm\" (UniqueName: \"kubernetes.io/projected/c83a300a-7a84-40ed-85fd-baa1a196f0f3-kube-api-access-7crcm\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:16:39.749300 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.749130 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-certificates\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.749300 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.749156 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:16:39.749300 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.749187 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-ca-trust-extracted\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.749469 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:39.749303 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 17:16:39.749469 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.749367 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5q7sz\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-kube-api-access-5q7sz\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.749469 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:39.749378 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls podName:c83a300a-7a84-40ed-85fd-baa1a196f0f3 nodeName:}" failed. No retries permitted until 2026-07-08 17:16:40.249355916 +0000 UTC m=+34.475338020 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls") pod "dns-default-mqrpr" (UID: "c83a300a-7a84-40ed-85fd-baa1a196f0f3") : secret "dns-default-metrics-tls" not found Jul 08 17:16:39.749469 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.749417 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c83a300a-7a84-40ed-85fd-baa1a196f0f3-tmp-dir\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:16:39.749469 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.749447 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/75d2bfa3-7826-4c84-affb-b589ad1fa15e-cert\") pod \"ingress-canary-7lmsl\" (UID: \"75d2bfa3-7826-4c84-affb-b589ad1fa15e\") " pod="openshift-ingress-canary/ingress-canary-7lmsl" Jul 08 17:16:39.749894 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.749482 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-trusted-ca\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.749894 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.749538 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-installation-pull-secrets\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.749894 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.749563 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/626e8de7-fd53-4204-8f73-9d1d41b32d19-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-njbnd\" (UID: \"626e8de7-fd53-4204-8f73-9d1d41b32d19\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:16:39.749894 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.749611 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glrhh\" (UniqueName: \"kubernetes.io/projected/75d2bfa3-7826-4c84-affb-b589ad1fa15e-kube-api-access-glrhh\") pod \"ingress-canary-7lmsl\" (UID: \"75d2bfa3-7826-4c84-affb-b589ad1fa15e\") " pod="openshift-ingress-canary/ingress-canary-7lmsl" Jul 08 17:16:39.749894 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.749772 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c83a300a-7a84-40ed-85fd-baa1a196f0f3-tmp-dir\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:16:39.749894 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.749769 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-image-registry-private-configuration\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.749894 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.749848 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c83a300a-7a84-40ed-85fd-baa1a196f0f3-config-volume\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:16:39.749894 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.749875 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-njbnd\" (UID: \"626e8de7-fd53-4204-8f73-9d1d41b32d19\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:16:39.750270 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.749913 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-ca-trust-extracted\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.750270 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:39.749993 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 17:16:39.750270 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:39.750006 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-699f54966c-dtkx2: secret "image-registry-tls" not found Jul 08 17:16:39.750270 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:39.750057 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls podName:5140e60b-db4c-4ab8-96ae-31b07cb7b21f nodeName:}" failed. No retries permitted until 2026-07-08 17:16:40.250041951 +0000 UTC m=+34.476024063 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls") pod "image-registry-699f54966c-dtkx2" (UID: "5140e60b-db4c-4ab8-96ae-31b07cb7b21f") : secret "image-registry-tls" not found Jul 08 17:16:39.750270 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.749917 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.750270 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.750091 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-bound-sa-token\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.750270 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.750095 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-certificates\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.750270 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:39.750185 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 17:16:39.750270 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:39.750221 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert podName:626e8de7-fd53-4204-8f73-9d1d41b32d19 nodeName:}" failed. No retries permitted until 2026-07-08 17:16:40.250210122 +0000 UTC m=+34.476192225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-njbnd" (UID: "626e8de7-fd53-4204-8f73-9d1d41b32d19") : secret "networking-console-plugin-cert" not found Jul 08 17:16:39.750624 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.750388 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c83a300a-7a84-40ed-85fd-baa1a196f0f3-config-volume\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:16:39.750624 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.750389 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/626e8de7-fd53-4204-8f73-9d1d41b32d19-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-njbnd\" (UID: \"626e8de7-fd53-4204-8f73-9d1d41b32d19\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:16:39.750624 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.750449 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-trusted-ca\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.753391 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.753365 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-image-registry-private-configuration\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.753474 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.753414 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-installation-pull-secrets\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.775157 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.775129 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7crcm\" (UniqueName: \"kubernetes.io/projected/c83a300a-7a84-40ed-85fd-baa1a196f0f3-kube-api-access-7crcm\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:16:39.780317 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.780294 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5q7sz\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-kube-api-access-5q7sz\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.784541 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.784519 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-bound-sa-token\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:39.850932 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.850892 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/75d2bfa3-7826-4c84-affb-b589ad1fa15e-cert\") pod \"ingress-canary-7lmsl\" (UID: \"75d2bfa3-7826-4c84-affb-b589ad1fa15e\") " pod="openshift-ingress-canary/ingress-canary-7lmsl" Jul 08 17:16:39.851113 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.850948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glrhh\" (UniqueName: \"kubernetes.io/projected/75d2bfa3-7826-4c84-affb-b589ad1fa15e-kube-api-access-glrhh\") pod \"ingress-canary-7lmsl\" (UID: \"75d2bfa3-7826-4c84-affb-b589ad1fa15e\") " pod="openshift-ingress-canary/ingress-canary-7lmsl" Jul 08 17:16:39.853283 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.853259 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/75d2bfa3-7826-4c84-affb-b589ad1fa15e-cert\") pod \"ingress-canary-7lmsl\" (UID: \"75d2bfa3-7826-4c84-affb-b589ad1fa15e\") " pod="openshift-ingress-canary/ingress-canary-7lmsl" Jul 08 17:16:39.860819 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.860800 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glrhh\" (UniqueName: \"kubernetes.io/projected/75d2bfa3-7826-4c84-affb-b589ad1fa15e-kube-api-access-glrhh\") pod \"ingress-canary-7lmsl\" (UID: \"75d2bfa3-7826-4c84-affb-b589ad1fa15e\") " pod="openshift-ingress-canary/ingress-canary-7lmsl" Jul 08 17:16:39.910417 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:39.910398 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7lmsl" Jul 08 17:16:40.087102 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:40.087073 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7lmsl"] Jul 08 17:16:40.090903 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:16:40.090878 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75d2bfa3_7826_4c84_affb_b589ad1fa15e.slice/crio-7d0fa7661e46e93df59dec4b8d626ce9b8a1d4e890c28db9273aa3dffbdc1f08 WatchSource:0}: Error finding container 7d0fa7661e46e93df59dec4b8d626ce9b8a1d4e890c28db9273aa3dffbdc1f08: Status 404 returned error can't find the container with id 7d0fa7661e46e93df59dec4b8d626ce9b8a1d4e890c28db9273aa3dffbdc1f08 Jul 08 17:16:40.255002 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:40.254965 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-njbnd\" (UID: \"626e8de7-fd53-4204-8f73-9d1d41b32d19\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:16:40.255252 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:40.255011 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:40.255252 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:40.255035 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:16:40.255252 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:40.255116 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 17:16:40.255252 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:40.255133 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 17:16:40.255252 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:40.255138 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 17:16:40.255252 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:40.255157 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-699f54966c-dtkx2: secret "image-registry-tls" not found Jul 08 17:16:40.255252 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:40.255188 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert podName:626e8de7-fd53-4204-8f73-9d1d41b32d19 nodeName:}" failed. No retries permitted until 2026-07-08 17:16:41.255172632 +0000 UTC m=+35.481154743 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-njbnd" (UID: "626e8de7-fd53-4204-8f73-9d1d41b32d19") : secret "networking-console-plugin-cert" not found Jul 08 17:16:40.255252 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:40.255203 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls podName:c83a300a-7a84-40ed-85fd-baa1a196f0f3 nodeName:}" failed. No retries permitted until 2026-07-08 17:16:41.255197399 +0000 UTC m=+35.481179497 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls") pod "dns-default-mqrpr" (UID: "c83a300a-7a84-40ed-85fd-baa1a196f0f3") : secret "dns-default-metrics-tls" not found Jul 08 17:16:40.255252 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:40.255215 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls podName:5140e60b-db4c-4ab8-96ae-31b07cb7b21f nodeName:}" failed. No retries permitted until 2026-07-08 17:16:41.255208637 +0000 UTC m=+35.481190734 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls") pod "image-registry-699f54966c-dtkx2" (UID: "5140e60b-db4c-4ab8-96ae-31b07cb7b21f") : secret "image-registry-tls" not found Jul 08 17:16:40.594971 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:40.594880 2578 generic.go:356] "Generic (PLEG): container finished" podID="f238c743-a6a4-4b86-b9ef-081f6b2ae9d1" containerID="fee8671c34512fc0e31e8270d27c5edade3224e6e9850b7397cd688a860383d9" exitCode=0 Jul 08 17:16:40.594971 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:40.594952 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2hjbv" event={"ID":"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1","Type":"ContainerDied","Data":"fee8671c34512fc0e31e8270d27c5edade3224e6e9850b7397cd688a860383d9"} Jul 08 17:16:40.596090 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:40.596029 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7lmsl" event={"ID":"75d2bfa3-7826-4c84-affb-b589ad1fa15e","Type":"ContainerStarted","Data":"7d0fa7661e46e93df59dec4b8d626ce9b8a1d4e890c28db9273aa3dffbdc1f08"} Jul 08 17:16:41.264419 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:41.264388 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-njbnd\" (UID: \"626e8de7-fd53-4204-8f73-9d1d41b32d19\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:16:41.264655 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:41.264428 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:41.264655 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:41.264452 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:16:41.264655 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:41.264546 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 17:16:41.264655 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:41.264549 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 17:16:41.264655 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:41.264553 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 17:16:41.264655 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:41.264612 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls podName:c83a300a-7a84-40ed-85fd-baa1a196f0f3 nodeName:}" failed. No retries permitted until 2026-07-08 17:16:43.264577589 +0000 UTC m=+37.490559687 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls") pod "dns-default-mqrpr" (UID: "c83a300a-7a84-40ed-85fd-baa1a196f0f3") : secret "dns-default-metrics-tls" not found Jul 08 17:16:41.264655 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:41.264618 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-699f54966c-dtkx2: secret "image-registry-tls" not found Jul 08 17:16:41.264655 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:41.264625 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert podName:626e8de7-fd53-4204-8f73-9d1d41b32d19 nodeName:}" failed. No retries permitted until 2026-07-08 17:16:43.264618627 +0000 UTC m=+37.490600725 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-njbnd" (UID: "626e8de7-fd53-4204-8f73-9d1d41b32d19") : secret "networking-console-plugin-cert" not found Jul 08 17:16:41.264655 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:41.264655 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls podName:5140e60b-db4c-4ab8-96ae-31b07cb7b21f nodeName:}" failed. No retries permitted until 2026-07-08 17:16:43.264636317 +0000 UTC m=+37.490618438 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls") pod "image-registry-699f54966c-dtkx2" (UID: "5140e60b-db4c-4ab8-96ae-31b07cb7b21f") : secret "image-registry-tls" not found Jul 08 17:16:41.388041 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:41.388009 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:16:41.388206 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:41.388105 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:16:41.388206 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:41.388184 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:16:41.393969 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:41.393943 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 17:16:41.394086 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:41.393947 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 17:16:41.394900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:41.394884 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xkss2\"" Jul 08 17:16:41.395139 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:41.395126 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 17:16:41.395184 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:41.395140 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8pnd7\"" Jul 08 17:16:41.395184 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:41.395140 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 17:16:43.284028 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:43.283826 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-njbnd\" (UID: \"626e8de7-fd53-4204-8f73-9d1d41b32d19\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:16:43.284498 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:43.284056 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:43.284498 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:43.284089 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:16:43.284498 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:43.283997 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 17:16:43.284498 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:43.284184 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert podName:626e8de7-fd53-4204-8f73-9d1d41b32d19 nodeName:}" failed. No retries permitted until 2026-07-08 17:16:47.284161856 +0000 UTC m=+41.510143970 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-njbnd" (UID: "626e8de7-fd53-4204-8f73-9d1d41b32d19") : secret "networking-console-plugin-cert" not found Jul 08 17:16:43.284498 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:43.284221 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 17:16:43.284498 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:43.284232 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 17:16:43.284498 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:43.284252 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-699f54966c-dtkx2: secret "image-registry-tls" not found Jul 08 17:16:43.284498 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:43.284293 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls podName:c83a300a-7a84-40ed-85fd-baa1a196f0f3 nodeName:}" failed. No retries permitted until 2026-07-08 17:16:47.284281037 +0000 UTC m=+41.510263135 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls") pod "dns-default-mqrpr" (UID: "c83a300a-7a84-40ed-85fd-baa1a196f0f3") : secret "dns-default-metrics-tls" not found Jul 08 17:16:43.284498 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:43.284307 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls podName:5140e60b-db4c-4ab8-96ae-31b07cb7b21f nodeName:}" failed. No retries permitted until 2026-07-08 17:16:47.284301393 +0000 UTC m=+41.510283491 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls") pod "image-registry-699f54966c-dtkx2" (UID: "5140e60b-db4c-4ab8-96ae-31b07cb7b21f") : secret "image-registry-tls" not found Jul 08 17:16:46.615734 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:46.615696 2578 generic.go:356] "Generic (PLEG): container finished" podID="f238c743-a6a4-4b86-b9ef-081f6b2ae9d1" containerID="a3910c2895c7cc3cec14e468b4b3b56d933a9217e07579d61183154a9d077f3d" exitCode=0 Jul 08 17:16:46.616157 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:46.615764 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2hjbv" event={"ID":"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1","Type":"ContainerDied","Data":"a3910c2895c7cc3cec14e468b4b3b56d933a9217e07579d61183154a9d077f3d"} Jul 08 17:16:46.617194 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:46.617171 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7lmsl" event={"ID":"75d2bfa3-7826-4c84-affb-b589ad1fa15e","Type":"ContainerStarted","Data":"44ed360105b0066bc1d5e45274f732b10f2afc7694ed63b556372d4bb9a288ef"} Jul 08 17:16:46.670052 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:46.669995 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7lmsl" podStartSLOduration=1.4467131100000001 podStartE2EDuration="7.669970095s" totalImagesPullingTime="6.223256985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:16:46.669896087 +0000 UTC m=+40.895878219" watchObservedRunningTime="2026-07-08 17:16:46.669970095 +0000 UTC m=+40.895952216" Jul 08 17:16:47.316643 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:47.316601 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:47.316826 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:47.316665 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:16:47.316826 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:47.316717 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 17:16:47.316826 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:47.316735 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-njbnd\" (UID: \"626e8de7-fd53-4204-8f73-9d1d41b32d19\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:16:47.316826 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:47.316740 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-699f54966c-dtkx2: secret "image-registry-tls" not found Jul 08 17:16:47.316826 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:47.316788 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 17:16:47.316826 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:47.316819 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 17:16:47.317086 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:47.316833 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls podName:5140e60b-db4c-4ab8-96ae-31b07cb7b21f nodeName:}" failed. No retries permitted until 2026-07-08 17:16:55.316814126 +0000 UTC m=+49.542796225 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls") pod "image-registry-699f54966c-dtkx2" (UID: "5140e60b-db4c-4ab8-96ae-31b07cb7b21f") : secret "image-registry-tls" not found Jul 08 17:16:47.317086 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:47.316850 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert podName:626e8de7-fd53-4204-8f73-9d1d41b32d19 nodeName:}" failed. No retries permitted until 2026-07-08 17:16:55.316841724 +0000 UTC m=+49.542823823 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-njbnd" (UID: "626e8de7-fd53-4204-8f73-9d1d41b32d19") : secret "networking-console-plugin-cert" not found Jul 08 17:16:47.317086 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:47.316863 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls podName:c83a300a-7a84-40ed-85fd-baa1a196f0f3 nodeName:}" failed. No retries permitted until 2026-07-08 17:16:55.316856269 +0000 UTC m=+49.542838369 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls") pod "dns-default-mqrpr" (UID: "c83a300a-7a84-40ed-85fd-baa1a196f0f3") : secret "dns-default-metrics-tls" not found Jul 08 17:16:47.622831 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:47.622744 2578 generic.go:356] "Generic (PLEG): container finished" podID="f238c743-a6a4-4b86-b9ef-081f6b2ae9d1" containerID="e2935d5a1ee70ac26f206a108fb4241aa03a7fe75ad4b66b1870efce6e12471b" exitCode=0 Jul 08 17:16:47.622831 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:47.622812 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2hjbv" event={"ID":"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1","Type":"ContainerDied","Data":"e2935d5a1ee70ac26f206a108fb4241aa03a7fe75ad4b66b1870efce6e12471b"} Jul 08 17:16:48.630042 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:48.630010 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2hjbv" event={"ID":"f238c743-a6a4-4b86-b9ef-081f6b2ae9d1","Type":"ContainerStarted","Data":"749045f5cb742038bf952a232431821bad60a29bf05a49cac0861e87cc8b7fcb"} Jul 08 17:16:48.672947 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:48.672897 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2hjbv" podStartSLOduration=4.037507533 podStartE2EDuration="42.67287989s" totalImagesPullingTime="38.635372357s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:16:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-08 17:16:48.672771752 +0000 UTC m=+42.898753873" watchObservedRunningTime="2026-07-08 17:16:48.67287989 +0000 UTC m=+42.898862003" Jul 08 17:16:55.379325 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:55.379285 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-njbnd\" (UID: \"626e8de7-fd53-4204-8f73-9d1d41b32d19\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:16:55.379766 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:55.379334 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:16:55.379766 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:16:55.379375 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:16:55.379766 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:55.379425 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 17:16:55.379766 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:55.379501 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert podName:626e8de7-fd53-4204-8f73-9d1d41b32d19 nodeName:}" failed. No retries permitted until 2026-07-08 17:17:11.379483437 +0000 UTC m=+65.605465550 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-njbnd" (UID: "626e8de7-fd53-4204-8f73-9d1d41b32d19") : secret "networking-console-plugin-cert" not found Jul 08 17:16:55.379766 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:55.379507 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 17:16:55.379766 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:55.379532 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 17:16:55.379766 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:55.379547 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-699f54966c-dtkx2: secret "image-registry-tls" not found Jul 08 17:16:55.379766 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:55.379566 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls podName:c83a300a-7a84-40ed-85fd-baa1a196f0f3 nodeName:}" failed. No retries permitted until 2026-07-08 17:17:11.379550921 +0000 UTC m=+65.605533020 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls") pod "dns-default-mqrpr" (UID: "c83a300a-7a84-40ed-85fd-baa1a196f0f3") : secret "dns-default-metrics-tls" not found Jul 08 17:16:55.379766 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:16:55.379641 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls podName:5140e60b-db4c-4ab8-96ae-31b07cb7b21f nodeName:}" failed. No retries permitted until 2026-07-08 17:17:11.379630623 +0000 UTC m=+65.605612720 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls") pod "image-registry-699f54966c-dtkx2" (UID: "5140e60b-db4c-4ab8-96ae-31b07cb7b21f") : secret "image-registry-tls" not found Jul 08 17:17:04.581842 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:04.581811 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jk4p4" Jul 08 17:17:10.986480 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:10.986434 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs\") pod \"network-metrics-daemon-8dk7z\" (UID: \"78604c5e-b53b-4761-b62e-b7f964af3e44\") " pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:17:10.989321 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:10.989297 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 17:17:10.997567 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:17:10.997535 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 17:17:10.997749 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:17:10.997701 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs podName:78604c5e-b53b-4761-b62e-b7f964af3e44 nodeName:}" failed. No retries permitted until 2026-07-08 17:18:14.997674828 +0000 UTC m=+129.223656930 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs") pod "network-metrics-daemon-8dk7z" (UID: "78604c5e-b53b-4761-b62e-b7f964af3e44") : secret "metrics-daemon-secret" not found Jul 08 17:17:11.087813 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:11.087767 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret\") pod \"global-pull-secret-syncer-pqz5t\" (UID: \"17ebcc8b-068a-42c0-b17f-ebae02312e5b\") " pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:17:11.090995 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:11.090973 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 17:17:11.101820 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:11.101788 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/17ebcc8b-068a-42c0-b17f-ebae02312e5b-original-pull-secret\") pod \"global-pull-secret-syncer-pqz5t\" (UID: \"17ebcc8b-068a-42c0-b17f-ebae02312e5b\") " pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:17:11.104657 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:11.104635 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-pqz5t" Jul 08 17:17:11.189175 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:11.189144 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmwzw\" (UniqueName: \"kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw\") pod \"network-check-target-7n9w9\" (UID: \"b7f1f18b-4d22-40ed-9184-95cc2ae301be\") " pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:17:11.192508 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:11.192486 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 17:17:11.202600 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:11.202565 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 17:17:11.212431 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:11.212404 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmwzw\" (UniqueName: \"kubernetes.io/projected/b7f1f18b-4d22-40ed-9184-95cc2ae301be-kube-api-access-mmwzw\") pod \"network-check-target-7n9w9\" (UID: \"b7f1f18b-4d22-40ed-9184-95cc2ae301be\") " pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:17:11.247017 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:11.246929 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-pqz5t"] Jul 08 17:17:11.251145 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:17:11.251115 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ebcc8b_068a_42c0_b17f_ebae02312e5b.slice/crio-fd0c1c89a6ea217a095c86dd9d31e485b21196db7ef75814e9dbb8599be21e64 WatchSource:0}: Error finding container fd0c1c89a6ea217a095c86dd9d31e485b21196db7ef75814e9dbb8599be21e64: Status 404 returned error can't find the container with id fd0c1c89a6ea217a095c86dd9d31e485b21196db7ef75814e9dbb8599be21e64 Jul 08 17:17:11.391088 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:11.391050 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-njbnd\" (UID: \"626e8de7-fd53-4204-8f73-9d1d41b32d19\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:17:11.391088 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:11.391100 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:17:11.391341 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:11.391126 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:17:11.391341 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:17:11.391219 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 17:17:11.391341 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:17:11.391271 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls podName:c83a300a-7a84-40ed-85fd-baa1a196f0f3 nodeName:}" failed. No retries permitted until 2026-07-08 17:17:43.391257976 +0000 UTC m=+97.617240074 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls") pod "dns-default-mqrpr" (UID: "c83a300a-7a84-40ed-85fd-baa1a196f0f3") : secret "dns-default-metrics-tls" not found Jul 08 17:17:11.391341 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:17:11.391216 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 17:17:11.391341 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:17:11.391313 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert podName:626e8de7-fd53-4204-8f73-9d1d41b32d19 nodeName:}" failed. No retries permitted until 2026-07-08 17:17:43.391306178 +0000 UTC m=+97.617288276 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-njbnd" (UID: "626e8de7-fd53-4204-8f73-9d1d41b32d19") : secret "networking-console-plugin-cert" not found Jul 08 17:17:11.391341 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:17:11.391221 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 17:17:11.391341 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:17:11.391332 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-699f54966c-dtkx2: secret "image-registry-tls" not found Jul 08 17:17:11.391580 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:17:11.391390 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls podName:5140e60b-db4c-4ab8-96ae-31b07cb7b21f nodeName:}" failed. No retries permitted until 2026-07-08 17:17:43.391376221 +0000 UTC m=+97.617358320 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls") pod "image-registry-699f54966c-dtkx2" (UID: "5140e60b-db4c-4ab8-96ae-31b07cb7b21f") : secret "image-registry-tls" not found Jul 08 17:17:11.402092 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:11.402068 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8pnd7\"" Jul 08 17:17:11.408912 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:11.408884 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:17:11.550231 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:11.550198 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7n9w9"] Jul 08 17:17:11.553258 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:17:11.553229 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7f1f18b_4d22_40ed_9184_95cc2ae301be.slice/crio-26ec6fa5e2b72ffdd3957185748f994fecdad6eb2b5ac54598ef6087d36051d7 WatchSource:0}: Error finding container 26ec6fa5e2b72ffdd3957185748f994fecdad6eb2b5ac54598ef6087d36051d7: Status 404 returned error can't find the container with id 26ec6fa5e2b72ffdd3957185748f994fecdad6eb2b5ac54598ef6087d36051d7 Jul 08 17:17:11.685830 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:11.685790 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pqz5t" event={"ID":"17ebcc8b-068a-42c0-b17f-ebae02312e5b","Type":"ContainerStarted","Data":"fd0c1c89a6ea217a095c86dd9d31e485b21196db7ef75814e9dbb8599be21e64"} Jul 08 17:17:11.686834 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:11.686811 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7n9w9" event={"ID":"b7f1f18b-4d22-40ed-9184-95cc2ae301be","Type":"ContainerStarted","Data":"26ec6fa5e2b72ffdd3957185748f994fecdad6eb2b5ac54598ef6087d36051d7"} Jul 08 17:17:15.698554 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:15.698520 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7n9w9" event={"ID":"b7f1f18b-4d22-40ed-9184-95cc2ae301be","Type":"ContainerStarted","Data":"f35adad734d982b190b3724a058d33157e7edfc5d99838308af0a6c6ac4c7932"} Jul 08 17:17:15.699028 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:15.698650 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:17:17.705240 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:17.705197 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-pqz5t" event={"ID":"17ebcc8b-068a-42c0-b17f-ebae02312e5b","Type":"ContainerStarted","Data":"ff1086688b02f32edd7c7779bc111c9d74ef792769639fe9c9f452622ff0210c"} Jul 08 17:17:17.729997 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:17.729947 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7n9w9" podStartSLOduration=68.28393343 podStartE2EDuration="1m11.72993181s" totalImagesPullingTime="3.44599838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:17:15.723165176 +0000 UTC m=+69.949147288" watchObservedRunningTime="2026-07-08 17:17:17.72993181 +0000 UTC m=+71.955913971" Jul 08 17:17:17.730431 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:17.730408 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-pqz5t" podStartSLOduration=65.444948292 podStartE2EDuration="1m11.730402937s" totalImagesPullingTime="6.285454645s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:17:17.729643197 +0000 UTC m=+71.955625338" watchObservedRunningTime="2026-07-08 17:17:17.730402937 +0000 UTC m=+71.956385056" Jul 08 17:17:34.990078 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:34.989971 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll"] Jul 08 17:17:34.992354 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:34.992339 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" Jul 08 17:17:34.995105 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:34.995083 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 08 17:17:34.995939 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:34.995924 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 08 17:17:34.996002 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:34.995937 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 08 17:17:34.996002 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:34.995924 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 08 17:17:35.007665 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:35.007646 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll"] Jul 08 17:17:35.059545 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:35.059517 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d310a6a5-98c5-4686-9f6e-38fadfe512be-tmp\") pod \"klusterlet-addon-workmgr-975c668fd-v8lll\" (UID: \"d310a6a5-98c5-4686-9f6e-38fadfe512be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" Jul 08 17:17:35.059687 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:35.059552 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh2dk\" (UniqueName: \"kubernetes.io/projected/d310a6a5-98c5-4686-9f6e-38fadfe512be-kube-api-access-nh2dk\") pod \"klusterlet-addon-workmgr-975c668fd-v8lll\" (UID: \"d310a6a5-98c5-4686-9f6e-38fadfe512be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" Jul 08 17:17:35.059687 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:35.059596 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d310a6a5-98c5-4686-9f6e-38fadfe512be-klusterlet-config\") pod \"klusterlet-addon-workmgr-975c668fd-v8lll\" (UID: \"d310a6a5-98c5-4686-9f6e-38fadfe512be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" Jul 08 17:17:35.160423 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:35.160399 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d310a6a5-98c5-4686-9f6e-38fadfe512be-tmp\") pod \"klusterlet-addon-workmgr-975c668fd-v8lll\" (UID: \"d310a6a5-98c5-4686-9f6e-38fadfe512be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" Jul 08 17:17:35.160492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:35.160431 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh2dk\" (UniqueName: \"kubernetes.io/projected/d310a6a5-98c5-4686-9f6e-38fadfe512be-kube-api-access-nh2dk\") pod \"klusterlet-addon-workmgr-975c668fd-v8lll\" (UID: \"d310a6a5-98c5-4686-9f6e-38fadfe512be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" Jul 08 17:17:35.160492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:35.160449 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d310a6a5-98c5-4686-9f6e-38fadfe512be-klusterlet-config\") pod \"klusterlet-addon-workmgr-975c668fd-v8lll\" (UID: \"d310a6a5-98c5-4686-9f6e-38fadfe512be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" Jul 08 17:17:35.160809 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:35.160790 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d310a6a5-98c5-4686-9f6e-38fadfe512be-tmp\") pod \"klusterlet-addon-workmgr-975c668fd-v8lll\" (UID: \"d310a6a5-98c5-4686-9f6e-38fadfe512be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" Jul 08 17:17:35.162836 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:35.162821 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d310a6a5-98c5-4686-9f6e-38fadfe512be-klusterlet-config\") pod \"klusterlet-addon-workmgr-975c668fd-v8lll\" (UID: \"d310a6a5-98c5-4686-9f6e-38fadfe512be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" Jul 08 17:17:35.169043 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:35.169026 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh2dk\" (UniqueName: \"kubernetes.io/projected/d310a6a5-98c5-4686-9f6e-38fadfe512be-kube-api-access-nh2dk\") pod \"klusterlet-addon-workmgr-975c668fd-v8lll\" (UID: \"d310a6a5-98c5-4686-9f6e-38fadfe512be\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" Jul 08 17:17:35.301281 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:35.301230 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" Jul 08 17:17:35.417637 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:35.417605 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll"] Jul 08 17:17:35.420470 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:17:35.420439 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd310a6a5_98c5_4686_9f6e_38fadfe512be.slice/crio-a7e691be627dfe285a9e11f47c11f63a2b0bfe59ecc372eeb429ba1ca78db7c7 WatchSource:0}: Error finding container a7e691be627dfe285a9e11f47c11f63a2b0bfe59ecc372eeb429ba1ca78db7c7: Status 404 returned error can't find the container with id a7e691be627dfe285a9e11f47c11f63a2b0bfe59ecc372eeb429ba1ca78db7c7 Jul 08 17:17:35.749205 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:35.749173 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" event={"ID":"d310a6a5-98c5-4686-9f6e-38fadfe512be","Type":"ContainerStarted","Data":"a7e691be627dfe285a9e11f47c11f63a2b0bfe59ecc372eeb429ba1ca78db7c7"} Jul 08 17:17:39.760606 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:39.760557 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" event={"ID":"d310a6a5-98c5-4686-9f6e-38fadfe512be","Type":"ContainerStarted","Data":"7bc5db8ec4096fe8091c8b4f2e938ca9fe8e4fef53667af0e0eb51da408cf06f"} Jul 08 17:17:39.760980 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:39.760772 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" Jul 08 17:17:39.762332 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:39.762311 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" Jul 08 17:17:39.777137 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:39.777095 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" podStartSLOduration=2.350033788 podStartE2EDuration="5.777081807s" totalImagesPullingTime="3.427048019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:17:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:17:39.776955277 +0000 UTC m=+94.002937397" watchObservedRunningTime="2026-07-08 17:17:39.777081807 +0000 UTC m=+94.003063924" Jul 08 17:17:43.421803 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:43.421705 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-njbnd\" (UID: \"626e8de7-fd53-4204-8f73-9d1d41b32d19\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:17:43.421803 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:43.421752 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:17:43.421803 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:43.421776 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:17:43.422227 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:17:43.421866 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 17:17:43.422227 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:17:43.421871 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 17:17:43.422227 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:17:43.421896 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-699f54966c-dtkx2: secret "image-registry-tls" not found Jul 08 17:17:43.422227 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:17:43.421922 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls podName:c83a300a-7a84-40ed-85fd-baa1a196f0f3 nodeName:}" failed. No retries permitted until 2026-07-08 17:18:47.421909833 +0000 UTC m=+161.647891930 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls") pod "dns-default-mqrpr" (UID: "c83a300a-7a84-40ed-85fd-baa1a196f0f3") : secret "dns-default-metrics-tls" not found Jul 08 17:17:43.422227 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:17:43.421871 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 17:17:43.422227 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:17:43.421936 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls podName:5140e60b-db4c-4ab8-96ae-31b07cb7b21f nodeName:}" failed. No retries permitted until 2026-07-08 17:18:47.421929712 +0000 UTC m=+161.647911811 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls") pod "image-registry-699f54966c-dtkx2" (UID: "5140e60b-db4c-4ab8-96ae-31b07cb7b21f") : secret "image-registry-tls" not found Jul 08 17:17:43.422227 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:17:43.421956 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert podName:626e8de7-fd53-4204-8f73-9d1d41b32d19 nodeName:}" failed. No retries permitted until 2026-07-08 17:18:47.421943277 +0000 UTC m=+161.647925375 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-njbnd" (UID: "626e8de7-fd53-4204-8f73-9d1d41b32d19") : secret "networking-console-plugin-cert" not found Jul 08 17:17:46.704102 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:17:46.704075 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7n9w9" Jul 08 17:18:15.050455 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:18:15.050418 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs\") pod \"network-metrics-daemon-8dk7z\" (UID: \"78604c5e-b53b-4761-b62e-b7f964af3e44\") " pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:18:15.050936 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:18:15.050556 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 17:18:15.050936 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:18:15.050645 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs podName:78604c5e-b53b-4761-b62e-b7f964af3e44 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:17.050630506 +0000 UTC m=+251.276612609 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs") pod "network-metrics-daemon-8dk7z" (UID: "78604c5e-b53b-4761-b62e-b7f964af3e44") : secret "metrics-daemon-secret" not found Jul 08 17:18:42.559461 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:18:42.559413 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" podUID="626e8de7-fd53-4204-8f73-9d1d41b32d19" Jul 08 17:18:42.574112 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:18:42.574077 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-699f54966c-dtkx2" podUID="5140e60b-db4c-4ab8-96ae-31b07cb7b21f" Jul 08 17:18:42.595775 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:18:42.595737 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-mqrpr" podUID="c83a300a-7a84-40ed-85fd-baa1a196f0f3" Jul 08 17:18:42.905840 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:18:42.905760 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:18:42.905996 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:18:42.905762 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:18:42.905996 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:18:42.905762 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mqrpr" Jul 08 17:18:44.409249 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:18:44.409208 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:18:47.489699 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:18:47.489666 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:18:47.490184 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:18:47.489724 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-njbnd\" (UID: \"626e8de7-fd53-4204-8f73-9d1d41b32d19\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:18:47.490184 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:18:47.489821 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 17:18:47.490184 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:18:47.489862 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 17:18:47.490184 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:18:47.489879 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls podName:c83a300a-7a84-40ed-85fd-baa1a196f0f3 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:49.48986347 +0000 UTC m=+283.715845567 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls") pod "dns-default-mqrpr" (UID: "c83a300a-7a84-40ed-85fd-baa1a196f0f3") : secret "dns-default-metrics-tls" not found Jul 08 17:18:47.490184 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:18:47.489911 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert podName:626e8de7-fd53-4204-8f73-9d1d41b32d19 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:49.489899025 +0000 UTC m=+283.715881124 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-njbnd" (UID: "626e8de7-fd53-4204-8f73-9d1d41b32d19") : secret "networking-console-plugin-cert" not found Jul 08 17:18:47.490184 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:18:47.489938 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:18:47.490184 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:18:47.489999 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 17:18:47.490184 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:18:47.490006 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-699f54966c-dtkx2: secret "image-registry-tls" not found Jul 08 17:18:47.490184 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:18:47.490034 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls podName:5140e60b-db4c-4ab8-96ae-31b07cb7b21f nodeName:}" failed. No retries permitted until 2026-07-08 17:20:49.49002685 +0000 UTC m=+283.716008948 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls") pod "image-registry-699f54966c-dtkx2" (UID: "5140e60b-db4c-4ab8-96ae-31b07cb7b21f") : secret "image-registry-tls" not found Jul 08 17:18:56.389480 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:18:56.389443 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:19:39.037973 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:19:39.037936 2578 generic.go:356] "Generic (PLEG): container finished" podID="d310a6a5-98c5-4686-9f6e-38fadfe512be" containerID="7bc5db8ec4096fe8091c8b4f2e938ca9fe8e4fef53667af0e0eb51da408cf06f" exitCode=1 Jul 08 17:19:39.038432 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:19:39.037994 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" event={"ID":"d310a6a5-98c5-4686-9f6e-38fadfe512be","Type":"ContainerDied","Data":"7bc5db8ec4096fe8091c8b4f2e938ca9fe8e4fef53667af0e0eb51da408cf06f"} Jul 08 17:19:39.038432 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:19:39.038388 2578 scope.go:117] "RemoveContainer" containerID="7bc5db8ec4096fe8091c8b4f2e938ca9fe8e4fef53667af0e0eb51da408cf06f" Jul 08 17:19:39.761656 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:19:39.761613 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" Jul 08 17:19:40.042112 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:19:40.042037 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" event={"ID":"d310a6a5-98c5-4686-9f6e-38fadfe512be","Type":"ContainerStarted","Data":"09161100acf604f9d002e2104c38de3ca9f49c6ed1b3cdd346b0870b42d7cb94"} Jul 08 17:19:40.042434 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:19:40.042228 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" Jul 08 17:19:40.042719 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:19:40.042701 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-975c668fd-v8lll" Jul 08 17:20:17.091521 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:17.091476 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs\") pod \"network-metrics-daemon-8dk7z\" (UID: \"78604c5e-b53b-4761-b62e-b7f964af3e44\") " pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:20:17.092029 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:17.091652 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 08 17:20:17.092029 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:17.091727 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs podName:78604c5e-b53b-4761-b62e-b7f964af3e44 nodeName:}" failed. No retries permitted until 2026-07-08 17:22:19.091710891 +0000 UTC m=+373.317692989 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs") pod "network-metrics-daemon-8dk7z" (UID: "78604c5e-b53b-4761-b62e-b7f964af3e44") : secret "metrics-daemon-secret" not found Jul 08 17:20:33.147473 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.147396 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-79f98bd59-6bd9x"] Jul 08 17:20:33.149232 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.149218 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:33.153129 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.153106 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 08 17:20:33.153408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.153389 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 08 17:20:33.153547 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.153530 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-ddzhp\"" Jul 08 17:20:33.154155 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.154058 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 08 17:20:33.154155 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.154144 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 17:20:33.156195 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.156169 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 08 17:20:33.156274 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.156227 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 17:20:33.171333 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.171304 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79f98bd59-6bd9x"] Jul 08 17:20:33.212280 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.212255 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:33.212371 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.212295 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-default-certificate\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:33.212371 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.212315 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-stats-auth\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:33.212448 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.212387 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvnt6\" (UniqueName: \"kubernetes.io/projected/ed90d90c-0640-40ee-9e52-ad99c028002a-kube-api-access-dvnt6\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:33.212448 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.212416 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:33.237293 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.237267 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs"] Jul 08 17:20:33.239435 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.239420 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz"] Jul 08 17:20:33.239566 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.239551 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs" Jul 08 17:20:33.241563 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.241544 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-vwfjt"] Jul 08 17:20:33.241679 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.241655 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" Jul 08 17:20:33.243556 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.243531 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.243735 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.243713 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jqhpk\"" Jul 08 17:20:33.243989 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.243974 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 08 17:20:33.244160 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.244141 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 17:20:33.244224 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.244208 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 08 17:20:33.244306 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.244223 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 08 17:20:33.244965 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.244949 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 08 17:20:33.245065 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.244961 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 17:20:33.245065 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.244971 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 08 17:20:33.245065 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.244970 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 08 17:20:33.245065 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.244950 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mqtcj\"" Jul 08 17:20:33.245940 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.245925 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 17:20:33.245940 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.245932 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 17:20:33.248047 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.248029 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 08 17:20:33.248233 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.248217 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 08 17:20:33.248376 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.248361 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-k76n5\"" Jul 08 17:20:33.253015 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.252995 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz"] Jul 08 17:20:33.253824 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.253805 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 08 17:20:33.256496 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.256477 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs"] Jul 08 17:20:33.266828 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.266806 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-vwfjt"] Jul 08 17:20:33.313503 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.313469 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77fc57aa-c23d-4c29-8529-4ca576c115b2-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.313706 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.313505 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:33.313706 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.313543 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-default-certificate\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:33.313706 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.313573 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx66w\" (UniqueName: \"kubernetes.io/projected/21677fb3-72b7-4a6d-acf9-47f6cabde040-kube-api-access-bx66w\") pod \"cluster-monitoring-operator-64786d9cbb-qdcgz\" (UID: \"21677fb3-72b7-4a6d-acf9-47f6cabde040\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" Jul 08 17:20:33.313706 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.313618 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zkcb\" (UniqueName: \"kubernetes.io/projected/77fc57aa-c23d-4c29-8529-4ca576c115b2-kube-api-access-6zkcb\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.313706 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:33.313686 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle podName:ed90d90c-0640-40ee-9e52-ad99c028002a nodeName:}" failed. No retries permitted until 2026-07-08 17:20:33.813665926 +0000 UTC m=+268.039648036 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle") pod "router-default-79f98bd59-6bd9x" (UID: "ed90d90c-0640-40ee-9e52-ad99c028002a") : configmap references non-existent config key: service-ca.crt Jul 08 17:20:33.313902 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.313748 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/21677fb3-72b7-4a6d-acf9-47f6cabde040-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-qdcgz\" (UID: \"21677fb3-72b7-4a6d-acf9-47f6cabde040\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" Jul 08 17:20:33.313902 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.313768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-stats-auth\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:33.313902 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.313784 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtrkt\" (UniqueName: \"kubernetes.io/projected/5b1cbe60-67d7-4c48-866a-73760293f220-kube-api-access-mtrkt\") pod \"kube-storage-version-migrator-operator-846bf668f7-qdcqs\" (UID: \"5b1cbe60-67d7-4c48-866a-73760293f220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs" Jul 08 17:20:33.313902 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.313807 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/77fc57aa-c23d-4c29-8529-4ca576c115b2-serving-cert\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.313902 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.313835 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77fc57aa-c23d-4c29-8529-4ca576c115b2-tmp\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.313902 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.313863 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvnt6\" (UniqueName: \"kubernetes.io/projected/ed90d90c-0640-40ee-9e52-ad99c028002a-kube-api-access-dvnt6\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:33.314126 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.313922 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b1cbe60-67d7-4c48-866a-73760293f220-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-qdcqs\" (UID: \"5b1cbe60-67d7-4c48-866a-73760293f220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs" Jul 08 17:20:33.314126 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.313952 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qdcgz\" (UID: \"21677fb3-72b7-4a6d-acf9-47f6cabde040\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" Jul 08 17:20:33.314126 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.313981 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77fc57aa-c23d-4c29-8529-4ca576c115b2-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.314126 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.314011 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/77fc57aa-c23d-4c29-8529-4ca576c115b2-snapshots\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.314126 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.314054 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:33.314126 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.314099 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5b1cbe60-67d7-4c48-866a-73760293f220-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-qdcqs\" (UID: \"5b1cbe60-67d7-4c48-866a-73760293f220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs" Jul 08 17:20:33.314319 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:33.314132 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 17:20:33.314319 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:33.314196 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs podName:ed90d90c-0640-40ee-9e52-ad99c028002a nodeName:}" failed. No retries permitted until 2026-07-08 17:20:33.814181079 +0000 UTC m=+268.040163190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs") pod "router-default-79f98bd59-6bd9x" (UID: "ed90d90c-0640-40ee-9e52-ad99c028002a") : secret "router-metrics-certs-default" not found Jul 08 17:20:33.316063 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.316041 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-default-certificate\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:33.316160 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.316048 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-stats-auth\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:33.321322 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.321302 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvnt6\" (UniqueName: \"kubernetes.io/projected/ed90d90c-0640-40ee-9e52-ad99c028002a-kube-api-access-dvnt6\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:33.414519 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.414420 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/21677fb3-72b7-4a6d-acf9-47f6cabde040-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-qdcgz\" (UID: \"21677fb3-72b7-4a6d-acf9-47f6cabde040\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" Jul 08 17:20:33.414519 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.414463 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtrkt\" (UniqueName: \"kubernetes.io/projected/5b1cbe60-67d7-4c48-866a-73760293f220-kube-api-access-mtrkt\") pod \"kube-storage-version-migrator-operator-846bf668f7-qdcqs\" (UID: \"5b1cbe60-67d7-4c48-866a-73760293f220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs" Jul 08 17:20:33.414519 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.414482 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/77fc57aa-c23d-4c29-8529-4ca576c115b2-serving-cert\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.414839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.414697 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77fc57aa-c23d-4c29-8529-4ca576c115b2-tmp\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.414839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.414741 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b1cbe60-67d7-4c48-866a-73760293f220-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-qdcqs\" (UID: \"5b1cbe60-67d7-4c48-866a-73760293f220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs" Jul 08 17:20:33.414839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.414766 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qdcgz\" (UID: \"21677fb3-72b7-4a6d-acf9-47f6cabde040\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" Jul 08 17:20:33.414839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.414796 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77fc57aa-c23d-4c29-8529-4ca576c115b2-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.414839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.414826 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/77fc57aa-c23d-4c29-8529-4ca576c115b2-snapshots\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.415143 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:33.414874 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 17:20:33.415143 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.414883 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5b1cbe60-67d7-4c48-866a-73760293f220-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-qdcqs\" (UID: \"5b1cbe60-67d7-4c48-866a-73760293f220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs" Jul 08 17:20:33.415143 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.414924 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77fc57aa-c23d-4c29-8529-4ca576c115b2-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.415143 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:33.414937 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls podName:21677fb3-72b7-4a6d-acf9-47f6cabde040 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:33.914917277 +0000 UTC m=+268.140899377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qdcgz" (UID: "21677fb3-72b7-4a6d-acf9-47f6cabde040") : secret "cluster-monitoring-operator-tls" not found Jul 08 17:20:33.415143 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.415015 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx66w\" (UniqueName: \"kubernetes.io/projected/21677fb3-72b7-4a6d-acf9-47f6cabde040-kube-api-access-bx66w\") pod \"cluster-monitoring-operator-64786d9cbb-qdcgz\" (UID: \"21677fb3-72b7-4a6d-acf9-47f6cabde040\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" Jul 08 17:20:33.415143 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.415051 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zkcb\" (UniqueName: \"kubernetes.io/projected/77fc57aa-c23d-4c29-8529-4ca576c115b2-kube-api-access-6zkcb\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.415449 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.415184 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/77fc57aa-c23d-4c29-8529-4ca576c115b2-tmp\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.415507 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.415453 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/77fc57aa-c23d-4c29-8529-4ca576c115b2-snapshots\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.415598 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.415556 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/21677fb3-72b7-4a6d-acf9-47f6cabde040-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-qdcgz\" (UID: \"21677fb3-72b7-4a6d-acf9-47f6cabde040\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" Jul 08 17:20:33.415768 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.415747 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5b1cbe60-67d7-4c48-866a-73760293f220-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-qdcqs\" (UID: \"5b1cbe60-67d7-4c48-866a-73760293f220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs" Jul 08 17:20:33.415974 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.415959 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77fc57aa-c23d-4c29-8529-4ca576c115b2-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.416135 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.416118 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/77fc57aa-c23d-4c29-8529-4ca576c115b2-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.417300 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.417283 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b1cbe60-67d7-4c48-866a-73760293f220-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-qdcqs\" (UID: \"5b1cbe60-67d7-4c48-866a-73760293f220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs" Jul 08 17:20:33.417379 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.417343 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/77fc57aa-c23d-4c29-8529-4ca576c115b2-serving-cert\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.423043 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.423020 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtrkt\" (UniqueName: \"kubernetes.io/projected/5b1cbe60-67d7-4c48-866a-73760293f220-kube-api-access-mtrkt\") pod \"kube-storage-version-migrator-operator-846bf668f7-qdcqs\" (UID: \"5b1cbe60-67d7-4c48-866a-73760293f220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs" Jul 08 17:20:33.423235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.423217 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx66w\" (UniqueName: \"kubernetes.io/projected/21677fb3-72b7-4a6d-acf9-47f6cabde040-kube-api-access-bx66w\") pod \"cluster-monitoring-operator-64786d9cbb-qdcgz\" (UID: \"21677fb3-72b7-4a6d-acf9-47f6cabde040\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" Jul 08 17:20:33.423449 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.423429 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zkcb\" (UniqueName: \"kubernetes.io/projected/77fc57aa-c23d-4c29-8529-4ca576c115b2-kube-api-access-6zkcb\") pod \"insights-operator-5c847dcfc4-vwfjt\" (UID: \"77fc57aa-c23d-4c29-8529-4ca576c115b2\") " pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.550657 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.550616 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs" Jul 08 17:20:33.563403 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.563375 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" Jul 08 17:20:33.685445 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.685414 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs"] Jul 08 17:20:33.688429 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:20:33.688401 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b1cbe60_67d7_4c48_866a_73760293f220.slice/crio-c2193ad309807221a48ae6da544dc8496ea95706d789ceff429584c0bb757a3b WatchSource:0}: Error finding container c2193ad309807221a48ae6da544dc8496ea95706d789ceff429584c0bb757a3b: Status 404 returned error can't find the container with id c2193ad309807221a48ae6da544dc8496ea95706d789ceff429584c0bb757a3b Jul 08 17:20:33.700943 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.700913 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-vwfjt"] Jul 08 17:20:33.708147 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:20:33.708121 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77fc57aa_c23d_4c29_8529_4ca576c115b2.slice/crio-16f01df8300aa1625313e6350a85a8d2f5b304a06ecc80a5282495a2c01d2e26 WatchSource:0}: Error finding container 16f01df8300aa1625313e6350a85a8d2f5b304a06ecc80a5282495a2c01d2e26: Status 404 returned error can't find the container with id 16f01df8300aa1625313e6350a85a8d2f5b304a06ecc80a5282495a2c01d2e26 Jul 08 17:20:33.819567 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.819525 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:33.819761 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.819600 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:33.819761 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:33.819689 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 17:20:33.819761 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:33.819728 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle podName:ed90d90c-0640-40ee-9e52-ad99c028002a nodeName:}" failed. No retries permitted until 2026-07-08 17:20:34.81971406 +0000 UTC m=+269.045696161 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle") pod "router-default-79f98bd59-6bd9x" (UID: "ed90d90c-0640-40ee-9e52-ad99c028002a") : configmap references non-existent config key: service-ca.crt Jul 08 17:20:33.819761 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:33.819743 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs podName:ed90d90c-0640-40ee-9e52-ad99c028002a nodeName:}" failed. No retries permitted until 2026-07-08 17:20:34.819736695 +0000 UTC m=+269.045718793 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs") pod "router-default-79f98bd59-6bd9x" (UID: "ed90d90c-0640-40ee-9e52-ad99c028002a") : secret "router-metrics-certs-default" not found Jul 08 17:20:33.920333 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:33.920287 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qdcgz\" (UID: \"21677fb3-72b7-4a6d-acf9-47f6cabde040\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" Jul 08 17:20:33.920512 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:33.920416 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 17:20:33.920512 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:33.920486 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls podName:21677fb3-72b7-4a6d-acf9-47f6cabde040 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:34.92046966 +0000 UTC m=+269.146451759 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qdcgz" (UID: "21677fb3-72b7-4a6d-acf9-47f6cabde040") : secret "cluster-monitoring-operator-tls" not found Jul 08 17:20:34.171742 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:34.171656 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" event={"ID":"77fc57aa-c23d-4c29-8529-4ca576c115b2","Type":"ContainerStarted","Data":"16f01df8300aa1625313e6350a85a8d2f5b304a06ecc80a5282495a2c01d2e26"} Jul 08 17:20:34.172829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:34.172805 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs" event={"ID":"5b1cbe60-67d7-4c48-866a-73760293f220","Type":"ContainerStarted","Data":"c2193ad309807221a48ae6da544dc8496ea95706d789ceff429584c0bb757a3b"} Jul 08 17:20:34.828918 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:34.828871 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:34.829101 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:34.828947 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:34.829101 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:34.829028 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 17:20:34.829101 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:34.829089 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle podName:ed90d90c-0640-40ee-9e52-ad99c028002a nodeName:}" failed. No retries permitted until 2026-07-08 17:20:36.82906971 +0000 UTC m=+271.055051811 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle") pod "router-default-79f98bd59-6bd9x" (UID: "ed90d90c-0640-40ee-9e52-ad99c028002a") : configmap references non-existent config key: service-ca.crt Jul 08 17:20:34.829273 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:34.829111 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs podName:ed90d90c-0640-40ee-9e52-ad99c028002a nodeName:}" failed. No retries permitted until 2026-07-08 17:20:36.8290994 +0000 UTC m=+271.055081500 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs") pod "router-default-79f98bd59-6bd9x" (UID: "ed90d90c-0640-40ee-9e52-ad99c028002a") : secret "router-metrics-certs-default" not found Jul 08 17:20:34.929559 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:34.929519 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qdcgz\" (UID: \"21677fb3-72b7-4a6d-acf9-47f6cabde040\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" Jul 08 17:20:34.929747 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:34.929677 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 17:20:34.929747 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:34.929743 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls podName:21677fb3-72b7-4a6d-acf9-47f6cabde040 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:36.929727754 +0000 UTC m=+271.155709852 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qdcgz" (UID: "21677fb3-72b7-4a6d-acf9-47f6cabde040") : secret "cluster-monitoring-operator-tls" not found Jul 08 17:20:36.848603 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:36.848562 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:36.848961 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:36.848634 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:36.848961 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:36.848738 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 17:20:36.848961 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:36.848756 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle podName:ed90d90c-0640-40ee-9e52-ad99c028002a nodeName:}" failed. No retries permitted until 2026-07-08 17:20:40.848739749 +0000 UTC m=+275.074721850 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle") pod "router-default-79f98bd59-6bd9x" (UID: "ed90d90c-0640-40ee-9e52-ad99c028002a") : configmap references non-existent config key: service-ca.crt Jul 08 17:20:36.848961 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:36.848801 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs podName:ed90d90c-0640-40ee-9e52-ad99c028002a nodeName:}" failed. No retries permitted until 2026-07-08 17:20:40.848785167 +0000 UTC m=+275.074767267 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs") pod "router-default-79f98bd59-6bd9x" (UID: "ed90d90c-0640-40ee-9e52-ad99c028002a") : secret "router-metrics-certs-default" not found Jul 08 17:20:36.950001 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:36.949958 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qdcgz\" (UID: \"21677fb3-72b7-4a6d-acf9-47f6cabde040\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" Jul 08 17:20:36.950158 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:36.950126 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 17:20:36.950215 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:36.950203 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls podName:21677fb3-72b7-4a6d-acf9-47f6cabde040 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:40.950183476 +0000 UTC m=+275.176165580 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qdcgz" (UID: "21677fb3-72b7-4a6d-acf9-47f6cabde040") : secret "cluster-monitoring-operator-tls" not found Jul 08 17:20:37.182799 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.182762 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" event={"ID":"77fc57aa-c23d-4c29-8529-4ca576c115b2","Type":"ContainerStarted","Data":"53644b7b31cb55c956add5cd983d2b901d1f031e13fec926fc9ccd8878484fc4"} Jul 08 17:20:37.184225 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.184192 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs" event={"ID":"5b1cbe60-67d7-4c48-866a-73760293f220","Type":"ContainerStarted","Data":"f749bf9b0517bdbc7020532d9cecba0c18cc80f691bbd005864f1ba8c5fd5e04"} Jul 08 17:20:37.199367 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.199299 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" podStartSLOduration=1.506194501 podStartE2EDuration="4.199281777s" totalImagesPullingTime="2.693087276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:20:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:37.198408473 +0000 UTC m=+271.424390617" watchObservedRunningTime="2026-07-08 17:20:37.199281777 +0000 UTC m=+271.425263896" Jul 08 17:20:37.213885 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.213823 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs" podStartSLOduration=1.5025552009999998 podStartE2EDuration="4.21379417s" totalImagesPullingTime="2.711238969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:20:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:37.213258444 +0000 UTC m=+271.439240565" watchObservedRunningTime="2026-07-08 17:20:37.21379417 +0000 UTC m=+271.439776293" Jul 08 17:20:37.575225 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.575143 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mdqd8"] Jul 08 17:20:37.578411 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.578390 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:37.582519 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.582473 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 17:20:37.582519 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.582500 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 17:20:37.582784 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.582768 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6zxwc\"" Jul 08 17:20:37.594489 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.594460 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mdqd8"] Jul 08 17:20:37.656945 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.656883 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:37.656945 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.656944 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbm9f\" (UniqueName: \"kubernetes.io/projected/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-api-access-vbm9f\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:37.657169 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.657033 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-data-volume\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:37.657169 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.657096 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-crio-socket\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:37.657169 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.657136 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:37.758291 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.758253 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-data-volume\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:37.758463 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.758309 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-crio-socket\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:37.758463 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.758336 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:37.758463 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.758382 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:37.758463 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.758438 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbm9f\" (UniqueName: \"kubernetes.io/projected/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-api-access-vbm9f\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:37.758463 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.758446 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-crio-socket\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:37.758726 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:37.758497 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 17:20:37.758726 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:37.758558 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 17:20:37.758726 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:37.758567 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls podName:f7a07c99-280f-4bd0-97ff-c662de6cc3a5 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:38.258549192 +0000 UTC m=+272.484531294 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mdqd8" (UID: "f7a07c99-280f-4bd0-97ff-c662de6cc3a5") : secret "insights-runtime-extractor-tls" not found Jul 08 17:20:37.758726 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:37.758656 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm podName:f7a07c99-280f-4bd0-97ff-c662de6cc3a5 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:38.258639466 +0000 UTC m=+272.484621568 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mdqd8" (UID: "f7a07c99-280f-4bd0-97ff-c662de6cc3a5") : configmap "kube-rbac-proxy" not found Jul 08 17:20:37.758861 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.758723 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-data-volume\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:37.767649 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:37.767620 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbm9f\" (UniqueName: \"kubernetes.io/projected/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-api-access-vbm9f\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:38.262413 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:38.262350 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:38.262915 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:38.262604 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 17:20:38.262915 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:38.262687 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:38.262915 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:38.262723 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls podName:f7a07c99-280f-4bd0-97ff-c662de6cc3a5 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:39.262700111 +0000 UTC m=+273.488682211 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mdqd8" (UID: "f7a07c99-280f-4bd0-97ff-c662de6cc3a5") : secret "insights-runtime-extractor-tls" not found Jul 08 17:20:38.262915 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:38.262757 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 17:20:38.262915 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:38.262800 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm podName:f7a07c99-280f-4bd0-97ff-c662de6cc3a5 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:39.26278887 +0000 UTC m=+273.488770968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mdqd8" (UID: "f7a07c99-280f-4bd0-97ff-c662de6cc3a5") : configmap "kube-rbac-proxy" not found Jul 08 17:20:39.269858 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:39.269805 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:39.270268 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:39.269915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:39.270268 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:39.269947 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 17:20:39.270268 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:39.270022 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 17:20:39.270268 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:39.270026 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm podName:f7a07c99-280f-4bd0-97ff-c662de6cc3a5 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:41.270010208 +0000 UTC m=+275.495992306 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mdqd8" (UID: "f7a07c99-280f-4bd0-97ff-c662de6cc3a5") : configmap "kube-rbac-proxy" not found Jul 08 17:20:39.270268 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:39.270073 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls podName:f7a07c99-280f-4bd0-97ff-c662de6cc3a5 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:41.270061818 +0000 UTC m=+275.496043916 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mdqd8" (UID: "f7a07c99-280f-4bd0-97ff-c662de6cc3a5") : secret "insights-runtime-extractor-tls" not found Jul 08 17:20:39.741441 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:39.741413 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xntds_abdf9cce-f248-42d1-8b24-1f03dcc28ae6/dns-node-resolver/0.log" Jul 08 17:20:40.341077 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.341049 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7555c_5d3f876c-9309-476f-b7ff-88c6afd11721/node-ca/0.log" Jul 08 17:20:40.439861 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.439826 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-sgxk4"] Jul 08 17:20:40.443173 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.443156 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-sgxk4" Jul 08 17:20:40.445603 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.445568 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 08 17:20:40.445747 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.445625 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 08 17:20:40.445851 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.445836 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 08 17:20:40.446813 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.446796 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-dt7g6\"" Jul 08 17:20:40.446917 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.446826 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 08 17:20:40.451989 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.451971 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-sgxk4"] Jul 08 17:20:40.581068 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.581025 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0621b1a4-cf2e-45dd-b862-631ec703fb8c-signing-key\") pod \"service-ca-758f599c5-sgxk4\" (UID: \"0621b1a4-cf2e-45dd-b862-631ec703fb8c\") " pod="openshift-service-ca/service-ca-758f599c5-sgxk4" Jul 08 17:20:40.581247 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.581100 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0621b1a4-cf2e-45dd-b862-631ec703fb8c-signing-cabundle\") pod \"service-ca-758f599c5-sgxk4\" (UID: \"0621b1a4-cf2e-45dd-b862-631ec703fb8c\") " pod="openshift-service-ca/service-ca-758f599c5-sgxk4" Jul 08 17:20:40.581247 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.581177 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v52s9\" (UniqueName: \"kubernetes.io/projected/0621b1a4-cf2e-45dd-b862-631ec703fb8c-kube-api-access-v52s9\") pod \"service-ca-758f599c5-sgxk4\" (UID: \"0621b1a4-cf2e-45dd-b862-631ec703fb8c\") " pod="openshift-service-ca/service-ca-758f599c5-sgxk4" Jul 08 17:20:40.681866 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.681836 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0621b1a4-cf2e-45dd-b862-631ec703fb8c-signing-key\") pod \"service-ca-758f599c5-sgxk4\" (UID: \"0621b1a4-cf2e-45dd-b862-631ec703fb8c\") " pod="openshift-service-ca/service-ca-758f599c5-sgxk4" Jul 08 17:20:40.681958 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.681905 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0621b1a4-cf2e-45dd-b862-631ec703fb8c-signing-cabundle\") pod \"service-ca-758f599c5-sgxk4\" (UID: \"0621b1a4-cf2e-45dd-b862-631ec703fb8c\") " pod="openshift-service-ca/service-ca-758f599c5-sgxk4" Jul 08 17:20:40.682010 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.681996 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v52s9\" (UniqueName: \"kubernetes.io/projected/0621b1a4-cf2e-45dd-b862-631ec703fb8c-kube-api-access-v52s9\") pod \"service-ca-758f599c5-sgxk4\" (UID: \"0621b1a4-cf2e-45dd-b862-631ec703fb8c\") " pod="openshift-service-ca/service-ca-758f599c5-sgxk4" Jul 08 17:20:40.682563 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.682538 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0621b1a4-cf2e-45dd-b862-631ec703fb8c-signing-cabundle\") pod \"service-ca-758f599c5-sgxk4\" (UID: \"0621b1a4-cf2e-45dd-b862-631ec703fb8c\") " pod="openshift-service-ca/service-ca-758f599c5-sgxk4" Jul 08 17:20:40.684183 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.684165 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0621b1a4-cf2e-45dd-b862-631ec703fb8c-signing-key\") pod \"service-ca-758f599c5-sgxk4\" (UID: \"0621b1a4-cf2e-45dd-b862-631ec703fb8c\") " pod="openshift-service-ca/service-ca-758f599c5-sgxk4" Jul 08 17:20:40.689415 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.689386 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v52s9\" (UniqueName: \"kubernetes.io/projected/0621b1a4-cf2e-45dd-b862-631ec703fb8c-kube-api-access-v52s9\") pod \"service-ca-758f599c5-sgxk4\" (UID: \"0621b1a4-cf2e-45dd-b862-631ec703fb8c\") " pod="openshift-service-ca/service-ca-758f599c5-sgxk4" Jul 08 17:20:40.752408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.752366 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-sgxk4" Jul 08 17:20:40.874417 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.874384 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-sgxk4"] Jul 08 17:20:40.877386 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:20:40.877359 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0621b1a4_cf2e_45dd_b862_631ec703fb8c.slice/crio-ee7883edb80e0b4988cd6c14cf934dc6e950e7420bd7563e575b29b27865e80a WatchSource:0}: Error finding container ee7883edb80e0b4988cd6c14cf934dc6e950e7420bd7563e575b29b27865e80a: Status 404 returned error can't find the container with id ee7883edb80e0b4988cd6c14cf934dc6e950e7420bd7563e575b29b27865e80a Jul 08 17:20:40.883944 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.883920 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:40.884051 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.884010 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:40.884116 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:40.884064 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 17:20:40.884159 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:40.884129 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs podName:ed90d90c-0640-40ee-9e52-ad99c028002a nodeName:}" failed. No retries permitted until 2026-07-08 17:20:48.884110241 +0000 UTC m=+283.110092340 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs") pod "router-default-79f98bd59-6bd9x" (UID: "ed90d90c-0640-40ee-9e52-ad99c028002a") : secret "router-metrics-certs-default" not found Jul 08 17:20:40.884204 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:40.884168 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle podName:ed90d90c-0640-40ee-9e52-ad99c028002a nodeName:}" failed. No retries permitted until 2026-07-08 17:20:48.884153896 +0000 UTC m=+283.110135994 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle") pod "router-default-79f98bd59-6bd9x" (UID: "ed90d90c-0640-40ee-9e52-ad99c028002a") : configmap references non-existent config key: service-ca.crt Jul 08 17:20:40.984708 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:40.984609 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qdcgz\" (UID: \"21677fb3-72b7-4a6d-acf9-47f6cabde040\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" Jul 08 17:20:40.984881 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:40.984761 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 17:20:40.984881 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:40.984824 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls podName:21677fb3-72b7-4a6d-acf9-47f6cabde040 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:48.984808339 +0000 UTC m=+283.210790442 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qdcgz" (UID: "21677fb3-72b7-4a6d-acf9-47f6cabde040") : secret "cluster-monitoring-operator-tls" not found Jul 08 17:20:41.142337 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:41.142308 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7lmsl_75d2bfa3-7826-4c84-affb-b589ad1fa15e/serve-healthcheck-canary/0.log" Jul 08 17:20:41.199350 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:41.199309 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-sgxk4" event={"ID":"0621b1a4-cf2e-45dd-b862-631ec703fb8c","Type":"ContainerStarted","Data":"ee7883edb80e0b4988cd6c14cf934dc6e950e7420bd7563e575b29b27865e80a"} Jul 08 17:20:41.287132 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:41.287035 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:41.287289 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:41.287216 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 17:20:41.287289 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:41.287265 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 17:20:41.287289 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:41.287220 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:41.287431 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:41.287287 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls podName:f7a07c99-280f-4bd0-97ff-c662de6cc3a5 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:45.28726778 +0000 UTC m=+279.513249907 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mdqd8" (UID: "f7a07c99-280f-4bd0-97ff-c662de6cc3a5") : secret "insights-runtime-extractor-tls" not found Jul 08 17:20:41.287431 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:41.287331 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm podName:f7a07c99-280f-4bd0-97ff-c662de6cc3a5 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:45.287307329 +0000 UTC m=+279.513289432 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mdqd8" (UID: "f7a07c99-280f-4bd0-97ff-c662de6cc3a5") : configmap "kube-rbac-proxy" not found Jul 08 17:20:44.210872 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:44.210835 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-sgxk4" event={"ID":"0621b1a4-cf2e-45dd-b862-631ec703fb8c","Type":"ContainerStarted","Data":"832185758c4af319eebf90ba90549d4fc3a33c5cee877bb9bde5e05d4a932e04"} Jul 08 17:20:44.230840 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:44.230789 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-sgxk4" podStartSLOduration=1.812297311 podStartE2EDuration="4.230775076s" totalImagesPullingTime="2.418477765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:20:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:44.230722278 +0000 UTC m=+278.456704400" watchObservedRunningTime="2026-07-08 17:20:44.230775076 +0000 UTC m=+278.456757195" Jul 08 17:20:45.323071 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:45.323023 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:45.323555 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:45.323140 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:45.323555 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:45.323188 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 17:20:45.323555 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:45.323263 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls podName:f7a07c99-280f-4bd0-97ff-c662de6cc3a5 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:53.323241622 +0000 UTC m=+287.549223734 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mdqd8" (UID: "f7a07c99-280f-4bd0-97ff-c662de6cc3a5") : secret "insights-runtime-extractor-tls" not found Jul 08 17:20:45.323555 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:45.323459 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 17:20:45.323555 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:45.323508 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm podName:f7a07c99-280f-4bd0-97ff-c662de6cc3a5 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:53.323492764 +0000 UTC m=+287.549474867 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mdqd8" (UID: "f7a07c99-280f-4bd0-97ff-c662de6cc3a5") : configmap "kube-rbac-proxy" not found Jul 08 17:20:45.906808 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:45.906766 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" podUID="626e8de7-fd53-4204-8f73-9d1d41b32d19" Jul 08 17:20:45.906808 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:45.906767 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-699f54966c-dtkx2" podUID="5140e60b-db4c-4ab8-96ae-31b07cb7b21f" Jul 08 17:20:45.907033 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:45.906766 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-mqrpr" podUID="c83a300a-7a84-40ed-85fd-baa1a196f0f3" Jul 08 17:20:46.217978 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:46.217943 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:20:46.218184 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:46.217956 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:20:46.218184 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:46.218118 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mqrpr" Jul 08 17:20:48.956925 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:48.956891 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:48.957461 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:48.956980 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:20:48.957461 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:48.957064 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 08 17:20:48.957461 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:48.957146 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs podName:ed90d90c-0640-40ee-9e52-ad99c028002a nodeName:}" failed. No retries permitted until 2026-07-08 17:21:04.957131213 +0000 UTC m=+299.183113315 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs") pod "router-default-79f98bd59-6bd9x" (UID: "ed90d90c-0640-40ee-9e52-ad99c028002a") : secret "router-metrics-certs-default" not found Jul 08 17:20:48.957461 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:48.957160 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle podName:ed90d90c-0640-40ee-9e52-ad99c028002a nodeName:}" failed. No retries permitted until 2026-07-08 17:21:04.957154354 +0000 UTC m=+299.183136452 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle") pod "router-default-79f98bd59-6bd9x" (UID: "ed90d90c-0640-40ee-9e52-ad99c028002a") : configmap references non-existent config key: service-ca.crt Jul 08 17:20:49.058111 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:49.058070 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qdcgz\" (UID: \"21677fb3-72b7-4a6d-acf9-47f6cabde040\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" Jul 08 17:20:49.058273 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:49.058220 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 08 17:20:49.058350 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:49.058299 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls podName:21677fb3-72b7-4a6d-acf9-47f6cabde040 nodeName:}" failed. No retries permitted until 2026-07-08 17:21:05.058279197 +0000 UTC m=+299.284261312 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-qdcgz" (UID: "21677fb3-72b7-4a6d-acf9-47f6cabde040") : secret "cluster-monitoring-operator-tls" not found Jul 08 17:20:49.561905 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:49.561867 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:20:49.562098 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:49.561944 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-njbnd\" (UID: \"626e8de7-fd53-4204-8f73-9d1d41b32d19\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:20:49.562098 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:49.561978 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:20:49.562207 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:49.562123 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 08 17:20:49.562207 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:49.562199 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert podName:626e8de7-fd53-4204-8f73-9d1d41b32d19 nodeName:}" failed. No retries permitted until 2026-07-08 17:22:51.562182864 +0000 UTC m=+405.788164964 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-njbnd" (UID: "626e8de7-fd53-4204-8f73-9d1d41b32d19") : secret "networking-console-plugin-cert" not found Jul 08 17:20:49.564318 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:49.564294 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c83a300a-7a84-40ed-85fd-baa1a196f0f3-metrics-tls\") pod \"dns-default-mqrpr\" (UID: \"c83a300a-7a84-40ed-85fd-baa1a196f0f3\") " pod="openshift-dns/dns-default-mqrpr" Jul 08 17:20:49.564391 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:49.564307 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls\") pod \"image-registry-699f54966c-dtkx2\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:20:49.822058 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:49.821970 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-tnxw9\"" Jul 08 17:20:49.822058 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:49.821970 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dpkhh\"" Jul 08 17:20:49.829874 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:49.829851 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:20:49.829997 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:49.829948 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mqrpr" Jul 08 17:20:49.968456 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:49.968385 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-699f54966c-dtkx2"] Jul 08 17:20:49.973980 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:20:49.973956 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5140e60b_db4c_4ab8_96ae_31b07cb7b21f.slice/crio-2285f8d7de8507a547f6f6c8f8b2a7bedc0116fcbe0a9d39e2454dd9800e733d WatchSource:0}: Error finding container 2285f8d7de8507a547f6f6c8f8b2a7bedc0116fcbe0a9d39e2454dd9800e733d: Status 404 returned error can't find the container with id 2285f8d7de8507a547f6f6c8f8b2a7bedc0116fcbe0a9d39e2454dd9800e733d Jul 08 17:20:49.982050 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:49.982030 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mqrpr"] Jul 08 17:20:49.984216 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:20:49.984194 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc83a300a_7a84_40ed_85fd_baa1a196f0f3.slice/crio-1d53da80bf1b369fbbad191a958c75814646e27532bfd9e3b9d9e9ca9e20bdb2 WatchSource:0}: Error finding container 1d53da80bf1b369fbbad191a958c75814646e27532bfd9e3b9d9e9ca9e20bdb2: Status 404 returned error can't find the container with id 1d53da80bf1b369fbbad191a958c75814646e27532bfd9e3b9d9e9ca9e20bdb2 Jul 08 17:20:50.232453 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:50.232410 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mqrpr" event={"ID":"c83a300a-7a84-40ed-85fd-baa1a196f0f3","Type":"ContainerStarted","Data":"1d53da80bf1b369fbbad191a958c75814646e27532bfd9e3b9d9e9ca9e20bdb2"} Jul 08 17:20:50.234020 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:50.233995 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-699f54966c-dtkx2" event={"ID":"5140e60b-db4c-4ab8-96ae-31b07cb7b21f","Type":"ContainerStarted","Data":"8dc25fab0c919275491636a2634284fb35ed28857ae5b7df5e2682992c76b2be"} Jul 08 17:20:50.234150 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:50.234037 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-699f54966c-dtkx2" event={"ID":"5140e60b-db4c-4ab8-96ae-31b07cb7b21f","Type":"ContainerStarted","Data":"2285f8d7de8507a547f6f6c8f8b2a7bedc0116fcbe0a9d39e2454dd9800e733d"} Jul 08 17:20:50.234150 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:50.234134 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:20:50.252352 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:50.252289 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-699f54966c-dtkx2" podStartSLOduration=283.252272607 podStartE2EDuration="4m43.252272607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:16:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:50.251917507 +0000 UTC m=+284.477899627" watchObservedRunningTime="2026-07-08 17:20:50.252272607 +0000 UTC m=+284.478254728" Jul 08 17:20:52.241934 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:52.241894 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mqrpr" event={"ID":"c83a300a-7a84-40ed-85fd-baa1a196f0f3","Type":"ContainerStarted","Data":"fd0e5241a640fa1f233763fe94a30a903aec0ff75818cda51d0ac4000352382a"} Jul 08 17:20:52.242329 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:52.241949 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mqrpr" event={"ID":"c83a300a-7a84-40ed-85fd-baa1a196f0f3","Type":"ContainerStarted","Data":"82a9f8fb34ea276b0328cc78b11f8af023a23c752f7b25d231033725df792ab3"} Jul 08 17:20:52.242329 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:52.242029 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mqrpr" Jul 08 17:20:52.261858 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:52.261805 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mqrpr" podStartSLOduration=251.577192108 podStartE2EDuration="4m13.261788001s" totalImagesPullingTime="1.684595893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:52.260870681 +0000 UTC m=+286.486852800" watchObservedRunningTime="2026-07-08 17:20:52.261788001 +0000 UTC m=+286.487770121" Jul 08 17:20:53.396508 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:53.396448 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:53.397042 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:53.396561 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:53.397042 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:53.396684 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 17:20:53.397042 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:53.396735 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls podName:f7a07c99-280f-4bd0-97ff-c662de6cc3a5 nodeName:}" failed. No retries permitted until 2026-07-08 17:21:09.39672255 +0000 UTC m=+303.622704647 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mdqd8" (UID: "f7a07c99-280f-4bd0-97ff-c662de6cc3a5") : secret "insights-runtime-extractor-tls" not found Jul 08 17:20:53.397196 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:20:53.397106 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:20:59.390845 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:20:59.390805 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-8dk7z" podUID="78604c5e-b53b-4761-b62e-b7f964af3e44" Jul 08 17:21:02.246679 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:02.246651 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mqrpr" Jul 08 17:21:04.987101 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:04.987064 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:21:04.987621 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:04.987124 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:21:04.987740 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:04.987723 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed90d90c-0640-40ee-9e52-ad99c028002a-service-ca-bundle\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:21:04.989408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:04.989388 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ed90d90c-0640-40ee-9e52-ad99c028002a-metrics-certs\") pod \"router-default-79f98bd59-6bd9x\" (UID: \"ed90d90c-0640-40ee-9e52-ad99c028002a\") " pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:21:05.087898 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:05.087841 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qdcgz\" (UID: \"21677fb3-72b7-4a6d-acf9-47f6cabde040\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" Jul 08 17:21:05.090461 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:05.090435 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/21677fb3-72b7-4a6d-acf9-47f6cabde040-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-qdcgz\" (UID: \"21677fb3-72b7-4a6d-acf9-47f6cabde040\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" Jul 08 17:21:05.258612 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:05.258486 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:21:05.357939 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:05.357901 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" Jul 08 17:21:05.394733 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:05.394413 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79f98bd59-6bd9x"] Jul 08 17:21:05.397463 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:21:05.397437 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded90d90c_0640_40ee_9e52_ad99c028002a.slice/crio-fc72e3e3ef19841a86a5dd65ced0ff6bab72428221129473d44b2b2c9f5e4634 WatchSource:0}: Error finding container fc72e3e3ef19841a86a5dd65ced0ff6bab72428221129473d44b2b2c9f5e4634: Status 404 returned error can't find the container with id fc72e3e3ef19841a86a5dd65ced0ff6bab72428221129473d44b2b2c9f5e4634 Jul 08 17:21:05.495109 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:05.495075 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz"] Jul 08 17:21:05.498559 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:21:05.498520 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21677fb3_72b7_4a6d_acf9_47f6cabde040.slice/crio-fd89ffc6573052b639c8db6db19f8740d9f707f9eb81fe14100c4753d5131ef4 WatchSource:0}: Error finding container fd89ffc6573052b639c8db6db19f8740d9f707f9eb81fe14100c4753d5131ef4: Status 404 returned error can't find the container with id fd89ffc6573052b639c8db6db19f8740d9f707f9eb81fe14100c4753d5131ef4 Jul 08 17:21:06.181370 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:06.181333 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 17:21:06.291831 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:06.291787 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79f98bd59-6bd9x" event={"ID":"ed90d90c-0640-40ee-9e52-ad99c028002a","Type":"ContainerStarted","Data":"837c6a64c7092ca7c77c8cba32e6562c32e074a997e72745bdd23065ff4eb91b"} Jul 08 17:21:06.292024 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:06.291888 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79f98bd59-6bd9x" event={"ID":"ed90d90c-0640-40ee-9e52-ad99c028002a","Type":"ContainerStarted","Data":"fc72e3e3ef19841a86a5dd65ced0ff6bab72428221129473d44b2b2c9f5e4634"} Jul 08 17:21:06.292940 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:06.292914 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" event={"ID":"21677fb3-72b7-4a6d-acf9-47f6cabde040","Type":"ContainerStarted","Data":"fd89ffc6573052b639c8db6db19f8740d9f707f9eb81fe14100c4753d5131ef4"} Jul 08 17:21:06.315288 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:06.315224 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-79f98bd59-6bd9x" podStartSLOduration=33.315207544 podStartE2EDuration="33.315207544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:20:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:21:06.313743221 +0000 UTC m=+300.539725340" watchObservedRunningTime="2026-07-08 17:21:06.315207544 +0000 UTC m=+300.541189722" Jul 08 17:21:07.259654 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:07.259612 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:21:07.262480 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:07.262461 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:21:07.297412 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:07.297384 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:21:07.298718 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:07.298696 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-79f98bd59-6bd9x" Jul 08 17:21:08.302693 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:08.302657 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" event={"ID":"21677fb3-72b7-4a6d-acf9-47f6cabde040","Type":"ContainerStarted","Data":"93ffb626b7c139e27abcf52c36c79bf3a8f037fa832d21c0720de573a5c71ccc"} Jul 08 17:21:08.318689 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:08.318634 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-qdcgz" podStartSLOduration=33.538794911 podStartE2EDuration="35.318615328s" totalImagesPullingTime="1.779820417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:20:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:21:08.317786692 +0000 UTC m=+302.543768869" watchObservedRunningTime="2026-07-08 17:21:08.318615328 +0000 UTC m=+302.544597448" Jul 08 17:21:09.423348 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:09.423316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:21:09.425850 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:09.425817 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f7a07c99-280f-4bd0-97ff-c662de6cc3a5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mdqd8\" (UID: \"f7a07c99-280f-4bd0-97ff-c662de6cc3a5\") " pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:21:09.695253 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:09.695219 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6zxwc\"" Jul 08 17:21:09.703397 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:09.703373 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mdqd8" Jul 08 17:21:09.834634 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:09.834548 2578 patch_prober.go:28] interesting pod/image-registry-699f54966c-dtkx2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 08 17:21:09.834825 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:09.834643 2578 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-699f54966c-dtkx2" podUID="5140e60b-db4c-4ab8-96ae-31b07cb7b21f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 08 17:21:09.847252 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:09.847224 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mdqd8"] Jul 08 17:21:09.850626 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:21:09.850601 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7a07c99_280f_4bd0_97ff_c662de6cc3a5.slice/crio-f501d4452c26a6e56244990af416e2ac76b00f726e1f4586e4ed02f495215266 WatchSource:0}: Error finding container f501d4452c26a6e56244990af416e2ac76b00f726e1f4586e4ed02f495215266: Status 404 returned error can't find the container with id f501d4452c26a6e56244990af416e2ac76b00f726e1f4586e4ed02f495215266 Jul 08 17:21:09.851944 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:09.851925 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 17:21:10.311720 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:10.311678 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mdqd8" event={"ID":"f7a07c99-280f-4bd0-97ff-c662de6cc3a5","Type":"ContainerStarted","Data":"f501d4452c26a6e56244990af416e2ac76b00f726e1f4586e4ed02f495215266"} Jul 08 17:21:11.242759 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:11.242725 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:21:11.845285 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:11.845249 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn"] Jul 08 17:21:11.856225 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:11.856203 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" Jul 08 17:21:11.858857 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:11.858832 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 17:21:11.858857 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:11.858871 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 17:21:11.859065 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:11.859004 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-nqqn2\"" Jul 08 17:21:11.859801 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:11.859781 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 17:21:11.862785 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:11.862714 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn"] Jul 08 17:21:11.947878 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:11.947847 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0ebb1aec-031c-407f-8b34-49b723a6f2d7-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-9jczn\" (UID: \"0ebb1aec-031c-407f-8b34-49b723a6f2d7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" Jul 08 17:21:11.948025 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:11.947887 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ebb1aec-031c-407f-8b34-49b723a6f2d7-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-9jczn\" (UID: \"0ebb1aec-031c-407f-8b34-49b723a6f2d7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" Jul 08 17:21:11.948025 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:11.947931 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78h5w\" (UniqueName: \"kubernetes.io/projected/0ebb1aec-031c-407f-8b34-49b723a6f2d7-kube-api-access-78h5w\") pod \"prometheus-operator-567f7b8b7c-9jczn\" (UID: \"0ebb1aec-031c-407f-8b34-49b723a6f2d7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" Jul 08 17:21:11.948130 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:11.948082 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ebb1aec-031c-407f-8b34-49b723a6f2d7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-9jczn\" (UID: \"0ebb1aec-031c-407f-8b34-49b723a6f2d7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" Jul 08 17:21:12.048928 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:12.048884 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ebb1aec-031c-407f-8b34-49b723a6f2d7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-9jczn\" (UID: \"0ebb1aec-031c-407f-8b34-49b723a6f2d7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" Jul 08 17:21:12.049080 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:12.048970 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0ebb1aec-031c-407f-8b34-49b723a6f2d7-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-9jczn\" (UID: \"0ebb1aec-031c-407f-8b34-49b723a6f2d7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" Jul 08 17:21:12.049080 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:12.048989 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ebb1aec-031c-407f-8b34-49b723a6f2d7-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-9jczn\" (UID: \"0ebb1aec-031c-407f-8b34-49b723a6f2d7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" Jul 08 17:21:12.049080 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:12.049030 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78h5w\" (UniqueName: \"kubernetes.io/projected/0ebb1aec-031c-407f-8b34-49b723a6f2d7-kube-api-access-78h5w\") pod \"prometheus-operator-567f7b8b7c-9jczn\" (UID: \"0ebb1aec-031c-407f-8b34-49b723a6f2d7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" Jul 08 17:21:12.049765 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:12.049737 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ebb1aec-031c-407f-8b34-49b723a6f2d7-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-9jczn\" (UID: \"0ebb1aec-031c-407f-8b34-49b723a6f2d7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" Jul 08 17:21:12.051450 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:12.051417 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ebb1aec-031c-407f-8b34-49b723a6f2d7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-9jczn\" (UID: \"0ebb1aec-031c-407f-8b34-49b723a6f2d7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" Jul 08 17:21:12.051805 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:12.051782 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0ebb1aec-031c-407f-8b34-49b723a6f2d7-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-9jczn\" (UID: \"0ebb1aec-031c-407f-8b34-49b723a6f2d7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" Jul 08 17:21:12.059252 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:12.059226 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78h5w\" (UniqueName: \"kubernetes.io/projected/0ebb1aec-031c-407f-8b34-49b723a6f2d7-kube-api-access-78h5w\") pod \"prometheus-operator-567f7b8b7c-9jczn\" (UID: \"0ebb1aec-031c-407f-8b34-49b723a6f2d7\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" Jul 08 17:21:12.168712 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:12.168624 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" Jul 08 17:21:12.388754 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:12.388720 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:21:13.603416 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:13.603389 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn"] Jul 08 17:21:13.605775 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:21:13.605752 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ebb1aec_031c_407f_8b34_49b723a6f2d7.slice/crio-98408cb418b86a77267ab53e633706bba205cfb7ec34df9e018aeadc2355e1e5 WatchSource:0}: Error finding container 98408cb418b86a77267ab53e633706bba205cfb7ec34df9e018aeadc2355e1e5: Status 404 returned error can't find the container with id 98408cb418b86a77267ab53e633706bba205cfb7ec34df9e018aeadc2355e1e5 Jul 08 17:21:14.327214 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:14.327186 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mdqd8" event={"ID":"f7a07c99-280f-4bd0-97ff-c662de6cc3a5","Type":"ContainerStarted","Data":"eb798deb6a3250c60b0467bf3aeda9d34c1e1e2635765a7c8e686fd1e69f7609"} Jul 08 17:21:14.328254 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:14.328231 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" event={"ID":"0ebb1aec-031c-407f-8b34-49b723a6f2d7","Type":"ContainerStarted","Data":"98408cb418b86a77267ab53e633706bba205cfb7ec34df9e018aeadc2355e1e5"} Jul 08 17:21:15.335174 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:15.335137 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mdqd8" event={"ID":"f7a07c99-280f-4bd0-97ff-c662de6cc3a5","Type":"ContainerStarted","Data":"cec8b3094e62bcedfea01cc8866fef6e176b46be2549668d9f5abf0897b5dfc1"} Jul 08 17:21:15.338155 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:15.338057 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" event={"ID":"0ebb1aec-031c-407f-8b34-49b723a6f2d7","Type":"ContainerStarted","Data":"70bdef78a85f392bf42ae8285bf7364500a0f865d2f3ef5ccc2f7f30cccb9f83"} Jul 08 17:21:15.338155 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:15.338109 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" event={"ID":"0ebb1aec-031c-407f-8b34-49b723a6f2d7","Type":"ContainerStarted","Data":"b243cca0ebc4c1104790d035f38d54b485b09edcaa32c77c9667ef05368c9a33"} Jul 08 17:21:15.356213 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:15.356155 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-9jczn" podStartSLOduration=2.772734396 podStartE2EDuration="4.356137664s" totalImagesPullingTime="1.583403268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:21:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:21:15.355297715 +0000 UTC m=+309.581279856" watchObservedRunningTime="2026-07-08 17:21:15.356137664 +0000 UTC m=+309.582119784" Jul 08 17:21:17.169413 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.169328 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq"] Jul 08 17:21:17.172705 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.172685 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" Jul 08 17:21:17.175176 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.175130 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-f9jb7\"" Jul 08 17:21:17.175393 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.175376 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 08 17:21:17.175867 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.175848 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 08 17:21:17.183393 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.183363 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq"] Jul 08 17:21:17.190223 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.190198 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rswcb"] Jul 08 17:21:17.193829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.193810 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.196214 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.196192 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 17:21:17.196448 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.196427 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 17:21:17.196604 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.196546 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 17:21:17.196835 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.196685 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p2wsh\"" Jul 08 17:21:17.293475 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.293449 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77dd845b-a2ed-4eb0-bb1b-30e46c4c340a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-r8spq\" (UID: \"77dd845b-a2ed-4eb0-bb1b-30e46c4c340a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" Jul 08 17:21:17.293623 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.293488 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd9cd1e8-2df1-4133-b1c1-54673fc66875-metrics-client-ca\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.293623 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.293517 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/77dd845b-a2ed-4eb0-bb1b-30e46c4c340a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-r8spq\" (UID: \"77dd845b-a2ed-4eb0-bb1b-30e46c4c340a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" Jul 08 17:21:17.293623 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.293569 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98wsd\" (UniqueName: \"kubernetes.io/projected/77dd845b-a2ed-4eb0-bb1b-30e46c4c340a-kube-api-access-98wsd\") pod \"openshift-state-metrics-54cf96dfcf-r8spq\" (UID: \"77dd845b-a2ed-4eb0-bb1b-30e46c4c340a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" Jul 08 17:21:17.293787 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.293635 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fd9cd1e8-2df1-4133-b1c1-54673fc66875-node-exporter-wtmp\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.293787 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.293666 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fd9cd1e8-2df1-4133-b1c1-54673fc66875-node-exporter-accelerators-collector-config\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.293787 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.293722 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fd9cd1e8-2df1-4133-b1c1-54673fc66875-node-exporter-textfile\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.293787 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.293759 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd9cd1e8-2df1-4133-b1c1-54673fc66875-sys\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.293943 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.293816 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fd9cd1e8-2df1-4133-b1c1-54673fc66875-root\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.293943 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.293873 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77dd845b-a2ed-4eb0-bb1b-30e46c4c340a-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-r8spq\" (UID: \"77dd845b-a2ed-4eb0-bb1b-30e46c4c340a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" Jul 08 17:21:17.294020 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.293953 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s52ql\" (UniqueName: \"kubernetes.io/projected/fd9cd1e8-2df1-4133-b1c1-54673fc66875-kube-api-access-s52ql\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.294020 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.293982 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fd9cd1e8-2df1-4133-b1c1-54673fc66875-node-exporter-tls\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.294020 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.294007 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd9cd1e8-2df1-4133-b1c1-54673fc66875-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.347452 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.347422 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mdqd8" event={"ID":"f7a07c99-280f-4bd0-97ff-c662de6cc3a5","Type":"ContainerStarted","Data":"8df780cfbfa8ea80f3b8b70020ac06b952bdf028b2c00ffa76e5a2dab2ce788e"} Jul 08 17:21:17.364908 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.364296 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mdqd8" podStartSLOduration=33.508375915 podStartE2EDuration="40.364279698s" totalImagesPullingTime="6.855903783s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:20:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:21:17.363953824 +0000 UTC m=+311.589935945" watchObservedRunningTime="2026-07-08 17:21:17.364279698 +0000 UTC m=+311.590261819" Jul 08 17:21:17.394441 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.394420 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fd9cd1e8-2df1-4133-b1c1-54673fc66875-node-exporter-tls\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.394545 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.394448 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd9cd1e8-2df1-4133-b1c1-54673fc66875-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.394545 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.394478 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77dd845b-a2ed-4eb0-bb1b-30e46c4c340a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-r8spq\" (UID: \"77dd845b-a2ed-4eb0-bb1b-30e46c4c340a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" Jul 08 17:21:17.394678 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.394542 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd9cd1e8-2df1-4133-b1c1-54673fc66875-metrics-client-ca\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.394678 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.394580 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/77dd845b-a2ed-4eb0-bb1b-30e46c4c340a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-r8spq\" (UID: \"77dd845b-a2ed-4eb0-bb1b-30e46c4c340a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" Jul 08 17:21:17.394678 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.394624 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98wsd\" (UniqueName: \"kubernetes.io/projected/77dd845b-a2ed-4eb0-bb1b-30e46c4c340a-kube-api-access-98wsd\") pod \"openshift-state-metrics-54cf96dfcf-r8spq\" (UID: \"77dd845b-a2ed-4eb0-bb1b-30e46c4c340a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" Jul 08 17:21:17.394678 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.394663 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fd9cd1e8-2df1-4133-b1c1-54673fc66875-node-exporter-wtmp\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.394873 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.394689 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fd9cd1e8-2df1-4133-b1c1-54673fc66875-node-exporter-accelerators-collector-config\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.394873 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.394760 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fd9cd1e8-2df1-4133-b1c1-54673fc66875-node-exporter-textfile\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.394873 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.394798 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd9cd1e8-2df1-4133-b1c1-54673fc66875-sys\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.394873 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.394867 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fd9cd1e8-2df1-4133-b1c1-54673fc66875-root\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.395066 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.394891 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77dd845b-a2ed-4eb0-bb1b-30e46c4c340a-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-r8spq\" (UID: \"77dd845b-a2ed-4eb0-bb1b-30e46c4c340a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" Jul 08 17:21:17.395066 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.394956 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s52ql\" (UniqueName: \"kubernetes.io/projected/fd9cd1e8-2df1-4133-b1c1-54673fc66875-kube-api-access-s52ql\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.395066 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.394973 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd9cd1e8-2df1-4133-b1c1-54673fc66875-sys\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.395211 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.395128 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fd9cd1e8-2df1-4133-b1c1-54673fc66875-node-exporter-wtmp\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.395268 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.395255 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd9cd1e8-2df1-4133-b1c1-54673fc66875-metrics-client-ca\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.395322 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.395265 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fd9cd1e8-2df1-4133-b1c1-54673fc66875-node-exporter-textfile\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.395461 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.395438 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fd9cd1e8-2df1-4133-b1c1-54673fc66875-root\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.395641 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.395618 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fd9cd1e8-2df1-4133-b1c1-54673fc66875-node-exporter-accelerators-collector-config\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.395989 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.395953 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77dd845b-a2ed-4eb0-bb1b-30e46c4c340a-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-r8spq\" (UID: \"77dd845b-a2ed-4eb0-bb1b-30e46c4c340a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" Jul 08 17:21:17.397171 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.397151 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77dd845b-a2ed-4eb0-bb1b-30e46c4c340a-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-r8spq\" (UID: \"77dd845b-a2ed-4eb0-bb1b-30e46c4c340a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" Jul 08 17:21:17.397706 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.397690 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/77dd845b-a2ed-4eb0-bb1b-30e46c4c340a-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-r8spq\" (UID: \"77dd845b-a2ed-4eb0-bb1b-30e46c4c340a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" Jul 08 17:21:17.398140 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.398116 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd9cd1e8-2df1-4133-b1c1-54673fc66875-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.398248 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.398231 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fd9cd1e8-2df1-4133-b1c1-54673fc66875-node-exporter-tls\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.410419 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.410388 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s52ql\" (UniqueName: \"kubernetes.io/projected/fd9cd1e8-2df1-4133-b1c1-54673fc66875-kube-api-access-s52ql\") pod \"node-exporter-rswcb\" (UID: \"fd9cd1e8-2df1-4133-b1c1-54673fc66875\") " pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.411921 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.411899 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98wsd\" (UniqueName: \"kubernetes.io/projected/77dd845b-a2ed-4eb0-bb1b-30e46c4c340a-kube-api-access-98wsd\") pod \"openshift-state-metrics-54cf96dfcf-r8spq\" (UID: \"77dd845b-a2ed-4eb0-bb1b-30e46c4c340a\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" Jul 08 17:21:17.484047 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.484027 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" Jul 08 17:21:17.505398 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.505371 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rswcb" Jul 08 17:21:17.516373 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:21:17.516340 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd9cd1e8_2df1_4133_b1c1_54673fc66875.slice/crio-b9c1c3614e503a655091ff8bc5823c71c1b2e4e4d9095dbf326eedc75478412c WatchSource:0}: Error finding container b9c1c3614e503a655091ff8bc5823c71c1b2e4e4d9095dbf326eedc75478412c: Status 404 returned error can't find the container with id b9c1c3614e503a655091ff8bc5823c71c1b2e4e4d9095dbf326eedc75478412c Jul 08 17:21:17.625309 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:17.625285 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq"] Jul 08 17:21:17.627545 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:21:17.627522 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77dd845b_a2ed_4eb0_bb1b_30e46c4c340a.slice/crio-8068b48f9b074e62702d294a2d03503440abc957aabe84c5f1e9a67876f0e670 WatchSource:0}: Error finding container 8068b48f9b074e62702d294a2d03503440abc957aabe84c5f1e9a67876f0e670: Status 404 returned error can't find the container with id 8068b48f9b074e62702d294a2d03503440abc957aabe84c5f1e9a67876f0e670 Jul 08 17:21:18.354125 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:18.354089 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" event={"ID":"77dd845b-a2ed-4eb0-bb1b-30e46c4c340a","Type":"ContainerStarted","Data":"bb0530735d63f6aedac6f59e12467de086c482e78950bff04b65f5ec5de5e759"} Jul 08 17:21:18.354541 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:18.354165 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" event={"ID":"77dd845b-a2ed-4eb0-bb1b-30e46c4c340a","Type":"ContainerStarted","Data":"52c08336bd67d678b7b588377c72a71a49b3b5967e216f32cd5c8fffec86f8c0"} Jul 08 17:21:18.354541 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:18.354195 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" event={"ID":"77dd845b-a2ed-4eb0-bb1b-30e46c4c340a","Type":"ContainerStarted","Data":"8068b48f9b074e62702d294a2d03503440abc957aabe84c5f1e9a67876f0e670"} Jul 08 17:21:18.355391 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:18.355342 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rswcb" event={"ID":"fd9cd1e8-2df1-4133-b1c1-54673fc66875","Type":"ContainerStarted","Data":"b9c1c3614e503a655091ff8bc5823c71c1b2e4e4d9095dbf326eedc75478412c"} Jul 08 17:21:19.360899 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:19.360813 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" event={"ID":"77dd845b-a2ed-4eb0-bb1b-30e46c4c340a","Type":"ContainerStarted","Data":"0020624e55ec3e2647e7010b09e77c7bb524f5fa3e44bcfb5e81c60afd9e5310"} Jul 08 17:21:19.362242 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:19.362220 2578 generic.go:356] "Generic (PLEG): container finished" podID="fd9cd1e8-2df1-4133-b1c1-54673fc66875" containerID="c9bb84482941c9430c06aa2926cc82b97a960dae2f74eba50159b5e7e328e427" exitCode=0 Jul 08 17:21:19.362341 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:19.362252 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rswcb" event={"ID":"fd9cd1e8-2df1-4133-b1c1-54673fc66875","Type":"ContainerDied","Data":"c9bb84482941c9430c06aa2926cc82b97a960dae2f74eba50159b5e7e328e427"} Jul 08 17:21:19.375604 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:19.375549 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-r8spq" podStartSLOduration=1.010172033 podStartE2EDuration="2.375533783s" totalImagesPullingTime="1.36536175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:21:19.37497724 +0000 UTC m=+313.600959381" watchObservedRunningTime="2026-07-08 17:21:19.375533783 +0000 UTC m=+313.601515903" Jul 08 17:21:20.368181 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:20.368100 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rswcb" event={"ID":"fd9cd1e8-2df1-4133-b1c1-54673fc66875","Type":"ContainerStarted","Data":"a7080eab6cfa663dee5b124be925956dddcd77c6773c76ef61edaa6d09b76fc7"} Jul 08 17:21:20.368682 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:20.368217 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rswcb" event={"ID":"fd9cd1e8-2df1-4133-b1c1-54673fc66875","Type":"ContainerStarted","Data":"c4602abcaba8130fa9d6a9796b41129d9c55df2820639dea418f629ca47392f7"} Jul 08 17:21:20.393028 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:20.392973 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rswcb" podStartSLOduration=2.288584774 podStartE2EDuration="3.392953929s" totalImagesPullingTime="1.027956796s" totalInitContainerRuntime="76.412359ms" isStatefulPod=true podCreationTimestamp="2026-07-08 17:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:21:20.39149479 +0000 UTC m=+314.617476910" watchObservedRunningTime="2026-07-08 17:21:20.392953929 +0000 UTC m=+314.618936050" Jul 08 17:21:23.983300 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:23.983268 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-699f54966c-dtkx2"] Jul 08 17:21:49.003515 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.003464 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-699f54966c-dtkx2" podUID="5140e60b-db4c-4ab8-96ae-31b07cb7b21f" containerName="registry" containerID="cri-o://8dc25fab0c919275491636a2634284fb35ed28857ae5b7df5e2682992c76b2be" gracePeriod=30 Jul 08 17:21:49.234829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.234806 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:21:49.346310 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.346237 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-certificates\") pod \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " Jul 08 17:21:49.346310 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.346314 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-installation-pull-secrets\") pod \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " Jul 08 17:21:49.346505 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.346347 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-bound-sa-token\") pod \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " Jul 08 17:21:49.346505 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.346371 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls\") pod \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " Jul 08 17:21:49.346505 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.346398 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5q7sz\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-kube-api-access-5q7sz\") pod \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " Jul 08 17:21:49.346505 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.346436 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-ca-trust-extracted\") pod \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " Jul 08 17:21:49.346505 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.346459 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-trusted-ca\") pod \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " Jul 08 17:21:49.346784 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.346514 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-image-registry-private-configuration\") pod \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\" (UID: \"5140e60b-db4c-4ab8-96ae-31b07cb7b21f\") " Jul 08 17:21:49.346784 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.346702 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5140e60b-db4c-4ab8-96ae-31b07cb7b21f" (UID: "5140e60b-db4c-4ab8-96ae-31b07cb7b21f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:49.346945 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.346911 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-certificates\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:21:49.347680 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.347646 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5140e60b-db4c-4ab8-96ae-31b07cb7b21f" (UID: "5140e60b-db4c-4ab8-96ae-31b07cb7b21f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:49.349225 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.349194 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5140e60b-db4c-4ab8-96ae-31b07cb7b21f" (UID: "5140e60b-db4c-4ab8-96ae-31b07cb7b21f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:21:49.349225 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.349212 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5140e60b-db4c-4ab8-96ae-31b07cb7b21f" (UID: "5140e60b-db4c-4ab8-96ae-31b07cb7b21f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:49.349389 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.349235 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5140e60b-db4c-4ab8-96ae-31b07cb7b21f" (UID: "5140e60b-db4c-4ab8-96ae-31b07cb7b21f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:21:49.349389 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.349328 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5140e60b-db4c-4ab8-96ae-31b07cb7b21f" (UID: "5140e60b-db4c-4ab8-96ae-31b07cb7b21f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:49.349500 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.349386 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-kube-api-access-5q7sz" (OuterVolumeSpecName: "kube-api-access-5q7sz") pod "5140e60b-db4c-4ab8-96ae-31b07cb7b21f" (UID: "5140e60b-db4c-4ab8-96ae-31b07cb7b21f"). InnerVolumeSpecName "kube-api-access-5q7sz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:21:49.355709 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.355685 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5140e60b-db4c-4ab8-96ae-31b07cb7b21f" (UID: "5140e60b-db4c-4ab8-96ae-31b07cb7b21f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:21:49.447935 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.447904 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-ca-trust-extracted\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:21:49.447935 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.447933 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-trusted-ca\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:21:49.448052 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.447943 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-image-registry-private-configuration\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:21:49.448052 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.447954 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-installation-pull-secrets\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:21:49.448052 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.447963 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-bound-sa-token\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:21:49.448052 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.447971 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-registry-tls\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:21:49.448052 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.447978 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5q7sz\" (UniqueName: \"kubernetes.io/projected/5140e60b-db4c-4ab8-96ae-31b07cb7b21f-kube-api-access-5q7sz\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:21:49.466274 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.466247 2578 generic.go:356] "Generic (PLEG): container finished" podID="5140e60b-db4c-4ab8-96ae-31b07cb7b21f" containerID="8dc25fab0c919275491636a2634284fb35ed28857ae5b7df5e2682992c76b2be" exitCode=0 Jul 08 17:21:49.466363 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.466286 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-699f54966c-dtkx2" Jul 08 17:21:49.466363 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.466309 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-699f54966c-dtkx2" event={"ID":"5140e60b-db4c-4ab8-96ae-31b07cb7b21f","Type":"ContainerDied","Data":"8dc25fab0c919275491636a2634284fb35ed28857ae5b7df5e2682992c76b2be"} Jul 08 17:21:49.466468 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.466366 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-699f54966c-dtkx2" event={"ID":"5140e60b-db4c-4ab8-96ae-31b07cb7b21f","Type":"ContainerDied","Data":"2285f8d7de8507a547f6f6c8f8b2a7bedc0116fcbe0a9d39e2454dd9800e733d"} Jul 08 17:21:49.466468 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.466396 2578 scope.go:117] "RemoveContainer" containerID="8dc25fab0c919275491636a2634284fb35ed28857ae5b7df5e2682992c76b2be" Jul 08 17:21:49.474310 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.474291 2578 scope.go:117] "RemoveContainer" containerID="8dc25fab0c919275491636a2634284fb35ed28857ae5b7df5e2682992c76b2be" Jul 08 17:21:49.474546 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:21:49.474527 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dc25fab0c919275491636a2634284fb35ed28857ae5b7df5e2682992c76b2be\": container with ID starting with 8dc25fab0c919275491636a2634284fb35ed28857ae5b7df5e2682992c76b2be not found: ID does not exist" containerID="8dc25fab0c919275491636a2634284fb35ed28857ae5b7df5e2682992c76b2be" Jul 08 17:21:49.474649 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.474553 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc25fab0c919275491636a2634284fb35ed28857ae5b7df5e2682992c76b2be"} err="failed to get container status \"8dc25fab0c919275491636a2634284fb35ed28857ae5b7df5e2682992c76b2be\": rpc error: code = NotFound desc = could not find container \"8dc25fab0c919275491636a2634284fb35ed28857ae5b7df5e2682992c76b2be\": container with ID starting with 8dc25fab0c919275491636a2634284fb35ed28857ae5b7df5e2682992c76b2be not found: ID does not exist" Jul 08 17:21:49.486417 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.486395 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-699f54966c-dtkx2"] Jul 08 17:21:49.490489 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:49.490469 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-699f54966c-dtkx2"] Jul 08 17:21:50.393261 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:50.393233 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5140e60b-db4c-4ab8-96ae-31b07cb7b21f" path="/var/lib/kubelet/pods/5140e60b-db4c-4ab8-96ae-31b07cb7b21f/volumes" Jul 08 17:21:52.477990 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:52.477927 2578 generic.go:356] "Generic (PLEG): container finished" podID="5b1cbe60-67d7-4c48-866a-73760293f220" containerID="f749bf9b0517bdbc7020532d9cecba0c18cc80f691bbd005864f1ba8c5fd5e04" exitCode=0 Jul 08 17:21:52.478371 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:52.478001 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs" event={"ID":"5b1cbe60-67d7-4c48-866a-73760293f220","Type":"ContainerDied","Data":"f749bf9b0517bdbc7020532d9cecba0c18cc80f691bbd005864f1ba8c5fd5e04"} Jul 08 17:21:52.478371 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:52.478302 2578 scope.go:117] "RemoveContainer" containerID="f749bf9b0517bdbc7020532d9cecba0c18cc80f691bbd005864f1ba8c5fd5e04" Jul 08 17:21:52.479614 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:52.479509 2578 generic.go:356] "Generic (PLEG): container finished" podID="77fc57aa-c23d-4c29-8529-4ca576c115b2" containerID="53644b7b31cb55c956add5cd983d2b901d1f031e13fec926fc9ccd8878484fc4" exitCode=0 Jul 08 17:21:52.479614 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:52.479536 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" event={"ID":"77fc57aa-c23d-4c29-8529-4ca576c115b2","Type":"ContainerDied","Data":"53644b7b31cb55c956add5cd983d2b901d1f031e13fec926fc9ccd8878484fc4"} Jul 08 17:21:52.479798 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:52.479784 2578 scope.go:117] "RemoveContainer" containerID="53644b7b31cb55c956add5cd983d2b901d1f031e13fec926fc9ccd8878484fc4" Jul 08 17:21:53.484178 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:53.484138 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-vwfjt" event={"ID":"77fc57aa-c23d-4c29-8529-4ca576c115b2","Type":"ContainerStarted","Data":"8fea59a8ed89651e7cebe548a00676685224f97d127d5a2a32a75e00d8684f88"} Jul 08 17:21:53.485865 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:21:53.485842 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-qdcqs" event={"ID":"5b1cbe60-67d7-4c48-866a-73760293f220","Type":"ContainerStarted","Data":"837806bc915abf31e401b258ec512ce81c612ec88a5672cc68f58bc7077d6b40"} Jul 08 17:22:19.163429 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:19.163372 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs\") pod \"network-metrics-daemon-8dk7z\" (UID: \"78604c5e-b53b-4761-b62e-b7f964af3e44\") " pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:22:19.165593 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:19.165564 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/78604c5e-b53b-4761-b62e-b7f964af3e44-metrics-certs\") pod \"network-metrics-daemon-8dk7z\" (UID: \"78604c5e-b53b-4761-b62e-b7f964af3e44\") " pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:22:19.292413 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:19.292382 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xkss2\"" Jul 08 17:22:19.300273 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:19.300254 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8dk7z" Jul 08 17:22:19.422569 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:19.422509 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8dk7z"] Jul 08 17:22:19.425259 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:22:19.425224 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78604c5e_b53b_4761_b62e_b7f964af3e44.slice/crio-15855251f38b2ad6047cf58f81487efe2f059df02ba65f4d94756dbb2cd9f7a2 WatchSource:0}: Error finding container 15855251f38b2ad6047cf58f81487efe2f059df02ba65f4d94756dbb2cd9f7a2: Status 404 returned error can't find the container with id 15855251f38b2ad6047cf58f81487efe2f059df02ba65f4d94756dbb2cd9f7a2 Jul 08 17:22:19.578662 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:19.578632 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8dk7z" event={"ID":"78604c5e-b53b-4761-b62e-b7f964af3e44","Type":"ContainerStarted","Data":"15855251f38b2ad6047cf58f81487efe2f059df02ba65f4d94756dbb2cd9f7a2"} Jul 08 17:22:21.587822 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:21.587792 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8dk7z" event={"ID":"78604c5e-b53b-4761-b62e-b7f964af3e44","Type":"ContainerStarted","Data":"01529517720a01a9b8b596c209d6986dd292f89d56b7722d592f25ebded0af3a"} Jul 08 17:22:21.588212 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:21.587840 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8dk7z" event={"ID":"78604c5e-b53b-4761-b62e-b7f964af3e44","Type":"ContainerStarted","Data":"e4057615287a7709b26a89beb4d8baadfa5d7ce81be7e30751007a9b077c5535"} Jul 08 17:22:21.603190 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:21.603140 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8dk7z" podStartSLOduration=374.263253117 podStartE2EDuration="6m15.603123818s" totalImagesPullingTime="1.339870701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:16:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:22:21.602855205 +0000 UTC m=+375.828837346" watchObservedRunningTime="2026-07-08 17:22:21.603123818 +0000 UTC m=+375.829105937" Jul 08 17:22:41.528490 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.528441 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-54f65bbbc-xtm5p"] Jul 08 17:22:41.529382 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.529351 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5140e60b-db4c-4ab8-96ae-31b07cb7b21f" containerName="registry" Jul 08 17:22:41.529382 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.529380 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5140e60b-db4c-4ab8-96ae-31b07cb7b21f" containerName="registry" Jul 08 17:22:41.529514 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.529468 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5140e60b-db4c-4ab8-96ae-31b07cb7b21f" containerName="registry" Jul 08 17:22:41.533626 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.533606 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.536153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.536134 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-sc69g\"" Jul 08 17:22:41.536650 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.536623 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 17:22:41.536830 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.536805 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 17:22:41.536943 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.536715 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 17:22:41.536999 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.536968 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 17:22:41.537305 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.537286 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 17:22:41.541706 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.541687 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 17:22:41.545121 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.545102 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-54f65bbbc-xtm5p"] Jul 08 17:22:41.633837 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.633798 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.633837 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.633841 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.634075 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.633869 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-secret-telemeter-client\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.634075 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.633940 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-telemeter-client-tls\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.634075 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.633961 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-metrics-client-ca\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.634075 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.633980 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcz4m\" (UniqueName: \"kubernetes.io/projected/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-kube-api-access-xcz4m\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.634075 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.634041 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-federate-client-tls\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.634075 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.634062 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-serving-certs-ca-bundle\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.735172 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.735135 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-federate-client-tls\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.735429 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.735411 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-serving-certs-ca-bundle\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.735633 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.735615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.735764 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.735749 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.735865 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.735853 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-secret-telemeter-client\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.736011 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.735998 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-telemeter-client-tls\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.736111 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.736099 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-metrics-client-ca\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.736203 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.736191 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcz4m\" (UniqueName: \"kubernetes.io/projected/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-kube-api-access-xcz4m\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.737146 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.737114 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-metrics-client-ca\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.737257 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.737188 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-telemeter-trusted-ca-bundle\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.737814 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.737791 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-serving-certs-ca-bundle\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.738519 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.738491 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-federate-client-tls\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.739504 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.739480 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-telemeter-client-tls\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.739610 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.739489 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.740042 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.740024 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-secret-telemeter-client\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.745575 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.745552 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcz4m\" (UniqueName: \"kubernetes.io/projected/42d242fa-3adb-4f60-b06d-9d0cc9dbbd45-kube-api-access-xcz4m\") pod \"telemeter-client-54f65bbbc-xtm5p\" (UID: \"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45\") " pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.846044 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.845958 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" Jul 08 17:22:41.994412 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:41.994387 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-54f65bbbc-xtm5p"] Jul 08 17:22:41.996862 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:22:41.996836 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42d242fa_3adb_4f60_b06d_9d0cc9dbbd45.slice/crio-8ba68a1237418dac85b2e09c9cd2950d99a6a2ea4917cf3eb2896f622b26e312 WatchSource:0}: Error finding container 8ba68a1237418dac85b2e09c9cd2950d99a6a2ea4917cf3eb2896f622b26e312: Status 404 returned error can't find the container with id 8ba68a1237418dac85b2e09c9cd2950d99a6a2ea4917cf3eb2896f622b26e312 Jul 08 17:22:42.661113 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:42.661070 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" event={"ID":"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45","Type":"ContainerStarted","Data":"8ba68a1237418dac85b2e09c9cd2950d99a6a2ea4917cf3eb2896f622b26e312"} Jul 08 17:22:44.670191 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:44.670149 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" event={"ID":"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45","Type":"ContainerStarted","Data":"d086065e62c4fab292c34d691de4a45d6c9c9326f3deef63e9cb57875261fd9e"} Jul 08 17:22:45.675836 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:45.675803 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" event={"ID":"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45","Type":"ContainerStarted","Data":"d0ddec8746fe81a46ec29ac10674da2bef1bd69c3034b968fd7304a6cadf3049"} Jul 08 17:22:45.676269 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:45.675858 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" event={"ID":"42d242fa-3adb-4f60-b06d-9d0cc9dbbd45","Type":"ContainerStarted","Data":"192645a49d7104a668c7576d9428f2de514c50faaa196098f1beb1cb980ced46"} Jul 08 17:22:45.701465 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:45.701411 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-54f65bbbc-xtm5p" podStartSLOduration=1.2958366049999999 podStartE2EDuration="4.701396136s" totalImagesPullingTime="3.405559531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:22:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:22:45.700690038 +0000 UTC m=+399.926672177" watchObservedRunningTime="2026-07-08 17:22:45.701396136 +0000 UTC m=+399.927378256" Jul 08 17:22:49.219602 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:22:49.219525 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" podUID="626e8de7-fd53-4204-8f73-9d1d41b32d19" Jul 08 17:22:49.695245 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:49.695216 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:22:51.623655 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:51.623615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-njbnd\" (UID: \"626e8de7-fd53-4204-8f73-9d1d41b32d19\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:22:51.625968 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:51.625949 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/626e8de7-fd53-4204-8f73-9d1d41b32d19-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-njbnd\" (UID: \"626e8de7-fd53-4204-8f73-9d1d41b32d19\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:22:51.799010 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:51.798978 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dmvf7\"" Jul 08 17:22:51.806821 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:51.806803 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" Jul 08 17:22:51.930467 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:51.930443 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-njbnd"] Jul 08 17:22:51.932765 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:22:51.932739 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod626e8de7_fd53_4204_8f73_9d1d41b32d19.slice/crio-2c62bf81caaebd802db82978eed246d58342f3d98223e84b4a7756ca12568637 WatchSource:0}: Error finding container 2c62bf81caaebd802db82978eed246d58342f3d98223e84b4a7756ca12568637: Status 404 returned error can't find the container with id 2c62bf81caaebd802db82978eed246d58342f3d98223e84b4a7756ca12568637 Jul 08 17:22:52.708577 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:52.708540 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" event={"ID":"626e8de7-fd53-4204-8f73-9d1d41b32d19","Type":"ContainerStarted","Data":"2c62bf81caaebd802db82978eed246d58342f3d98223e84b4a7756ca12568637"} Jul 08 17:22:53.713102 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:53.713070 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" event={"ID":"626e8de7-fd53-4204-8f73-9d1d41b32d19","Type":"ContainerStarted","Data":"0e84d0edc72a6f0e9907c9cc8976935592d55ccd0b89416e84ad1168208342b1"} Jul 08 17:22:53.735779 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:22:53.735730 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-njbnd" podStartSLOduration=382.423486318 podStartE2EDuration="6m23.735714675s" totalImagesPullingTime="1.312228357s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:16:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:22:53.734799954 +0000 UTC m=+407.960782074" watchObservedRunningTime="2026-07-08 17:22:53.735714675 +0000 UTC m=+407.961696796" Jul 08 17:24:48.257048 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:48.257003 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc"] Jul 08 17:24:48.262352 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:48.262326 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" Jul 08 17:24:48.265029 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:48.265002 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 17:24:48.265029 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:48.265002 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 17:24:48.265264 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:48.265144 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jhpsm\"" Jul 08 17:24:48.267653 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:48.267629 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc"] Jul 08 17:24:48.290725 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:48.290692 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc\" (UID: \"d3f717fb-b12b-47a1-be8c-a094a2cd36d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" Jul 08 17:24:48.290725 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:48.290734 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc\" (UID: \"d3f717fb-b12b-47a1-be8c-a094a2cd36d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" Jul 08 17:24:48.290948 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:48.290762 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xc29\" (UniqueName: \"kubernetes.io/projected/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-kube-api-access-2xc29\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc\" (UID: \"d3f717fb-b12b-47a1-be8c-a094a2cd36d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" Jul 08 17:24:48.391324 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:48.391286 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xc29\" (UniqueName: \"kubernetes.io/projected/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-kube-api-access-2xc29\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc\" (UID: \"d3f717fb-b12b-47a1-be8c-a094a2cd36d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" Jul 08 17:24:48.391503 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:48.391399 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc\" (UID: \"d3f717fb-b12b-47a1-be8c-a094a2cd36d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" Jul 08 17:24:48.391503 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:48.391441 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc\" (UID: \"d3f717fb-b12b-47a1-be8c-a094a2cd36d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" Jul 08 17:24:48.391853 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:48.391834 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc\" (UID: \"d3f717fb-b12b-47a1-be8c-a094a2cd36d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" Jul 08 17:24:48.391915 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:48.391852 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc\" (UID: \"d3f717fb-b12b-47a1-be8c-a094a2cd36d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" Jul 08 17:24:48.400135 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:48.400104 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xc29\" (UniqueName: \"kubernetes.io/projected/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-kube-api-access-2xc29\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc\" (UID: \"d3f717fb-b12b-47a1-be8c-a094a2cd36d2\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" Jul 08 17:24:48.573165 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:48.573071 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" Jul 08 17:24:48.710981 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:48.710945 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc"] Jul 08 17:24:48.714898 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:24:48.714870 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f717fb_b12b_47a1_be8c_a094a2cd36d2.slice/crio-01ddf78fc3b89de82ea514baa5edaed20b6cce4a956390923dec9648d7961cde WatchSource:0}: Error finding container 01ddf78fc3b89de82ea514baa5edaed20b6cce4a956390923dec9648d7961cde: Status 404 returned error can't find the container with id 01ddf78fc3b89de82ea514baa5edaed20b6cce4a956390923dec9648d7961cde Jul 08 17:24:49.136708 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:49.136672 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" event={"ID":"d3f717fb-b12b-47a1-be8c-a094a2cd36d2","Type":"ContainerStarted","Data":"01ddf78fc3b89de82ea514baa5edaed20b6cce4a956390923dec9648d7961cde"} Jul 08 17:24:58.179966 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:58.179931 2578 generic.go:356] "Generic (PLEG): container finished" podID="d3f717fb-b12b-47a1-be8c-a094a2cd36d2" containerID="2bc0212b7c8eedadce626bc4ed8e380161e1ef428699a74621f04d6ab8f3ac4f" exitCode=0 Jul 08 17:24:58.180364 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:24:58.180019 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" event={"ID":"d3f717fb-b12b-47a1-be8c-a094a2cd36d2","Type":"ContainerDied","Data":"2bc0212b7c8eedadce626bc4ed8e380161e1ef428699a74621f04d6ab8f3ac4f"} Jul 08 17:25:00.190072 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:00.190038 2578 generic.go:356] "Generic (PLEG): container finished" podID="d3f717fb-b12b-47a1-be8c-a094a2cd36d2" containerID="f7c213211f4c7d73de260d28a9f6eed466c1193efb90a935a2c8e2493feaffa7" exitCode=0 Jul 08 17:25:00.190514 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:00.190121 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" event={"ID":"d3f717fb-b12b-47a1-be8c-a094a2cd36d2","Type":"ContainerDied","Data":"f7c213211f4c7d73de260d28a9f6eed466c1193efb90a935a2c8e2493feaffa7"} Jul 08 17:25:10.233017 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:10.232972 2578 generic.go:356] "Generic (PLEG): container finished" podID="d3f717fb-b12b-47a1-be8c-a094a2cd36d2" containerID="a36e6d8aeb157740df4cb530b727a5540222a1619d85e887159956227016cc4f" exitCode=0 Jul 08 17:25:10.233396 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:10.233057 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" event={"ID":"d3f717fb-b12b-47a1-be8c-a094a2cd36d2","Type":"ContainerDied","Data":"a36e6d8aeb157740df4cb530b727a5540222a1619d85e887159956227016cc4f"} Jul 08 17:25:11.386741 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:11.386702 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" Jul 08 17:25:11.490386 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:11.490272 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-bundle\") pod \"d3f717fb-b12b-47a1-be8c-a094a2cd36d2\" (UID: \"d3f717fb-b12b-47a1-be8c-a094a2cd36d2\") " Jul 08 17:25:11.490561 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:11.490384 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-util\") pod \"d3f717fb-b12b-47a1-be8c-a094a2cd36d2\" (UID: \"d3f717fb-b12b-47a1-be8c-a094a2cd36d2\") " Jul 08 17:25:11.490561 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:11.490415 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xc29\" (UniqueName: \"kubernetes.io/projected/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-kube-api-access-2xc29\") pod \"d3f717fb-b12b-47a1-be8c-a094a2cd36d2\" (UID: \"d3f717fb-b12b-47a1-be8c-a094a2cd36d2\") " Jul 08 17:25:11.490962 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:11.490938 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-bundle" (OuterVolumeSpecName: "bundle") pod "d3f717fb-b12b-47a1-be8c-a094a2cd36d2" (UID: "d3f717fb-b12b-47a1-be8c-a094a2cd36d2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:25:11.492654 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:11.492622 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-kube-api-access-2xc29" (OuterVolumeSpecName: "kube-api-access-2xc29") pod "d3f717fb-b12b-47a1-be8c-a094a2cd36d2" (UID: "d3f717fb-b12b-47a1-be8c-a094a2cd36d2"). InnerVolumeSpecName "kube-api-access-2xc29". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:25:11.495612 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:11.495569 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-util" (OuterVolumeSpecName: "util") pod "d3f717fb-b12b-47a1-be8c-a094a2cd36d2" (UID: "d3f717fb-b12b-47a1-be8c-a094a2cd36d2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:25:11.591936 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:11.591884 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:25:11.591936 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:11.591927 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-util\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:25:11.591936 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:11.591936 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xc29\" (UniqueName: \"kubernetes.io/projected/d3f717fb-b12b-47a1-be8c-a094a2cd36d2-kube-api-access-2xc29\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:25:12.242718 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:12.242683 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" event={"ID":"d3f717fb-b12b-47a1-be8c-a094a2cd36d2","Type":"ContainerDied","Data":"01ddf78fc3b89de82ea514baa5edaed20b6cce4a956390923dec9648d7961cde"} Jul 08 17:25:12.242718 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:12.242719 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ngwgc" Jul 08 17:25:12.242969 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:12.242731 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01ddf78fc3b89de82ea514baa5edaed20b6cce4a956390923dec9648d7961cde" Jul 08 17:25:16.293519 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.293481 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-krb8z"] Jul 08 17:25:16.294006 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.293801 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3f717fb-b12b-47a1-be8c-a094a2cd36d2" containerName="util" Jul 08 17:25:16.294006 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.293813 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3f717fb-b12b-47a1-be8c-a094a2cd36d2" containerName="util" Jul 08 17:25:16.294006 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.293828 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3f717fb-b12b-47a1-be8c-a094a2cd36d2" containerName="pull" Jul 08 17:25:16.294006 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.293835 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3f717fb-b12b-47a1-be8c-a094a2cd36d2" containerName="pull" Jul 08 17:25:16.294006 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.293845 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3f717fb-b12b-47a1-be8c-a094a2cd36d2" containerName="extract" Jul 08 17:25:16.294006 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.293850 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3f717fb-b12b-47a1-be8c-a094a2cd36d2" containerName="extract" Jul 08 17:25:16.294006 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.293895 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3f717fb-b12b-47a1-be8c-a094a2cd36d2" containerName="extract" Jul 08 17:25:16.295941 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.295923 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-krb8z" Jul 08 17:25:16.298394 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.298367 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 08 17:25:16.298541 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.298476 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 08 17:25:16.298831 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.298811 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-pvwdj\"" Jul 08 17:25:16.309162 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.309125 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-krb8z"] Jul 08 17:25:16.431251 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.431214 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44a1876e-47a2-4c9c-ba62-a291294508ef-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-krb8z\" (UID: \"44a1876e-47a2-4c9c-ba62-a291294508ef\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-krb8z" Jul 08 17:25:16.431433 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.431339 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5rdx\" (UniqueName: \"kubernetes.io/projected/44a1876e-47a2-4c9c-ba62-a291294508ef-kube-api-access-t5rdx\") pod \"cert-manager-operator-controller-manager-55c444bf86-krb8z\" (UID: \"44a1876e-47a2-4c9c-ba62-a291294508ef\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-krb8z" Jul 08 17:25:16.532190 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.532154 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5rdx\" (UniqueName: \"kubernetes.io/projected/44a1876e-47a2-4c9c-ba62-a291294508ef-kube-api-access-t5rdx\") pod \"cert-manager-operator-controller-manager-55c444bf86-krb8z\" (UID: \"44a1876e-47a2-4c9c-ba62-a291294508ef\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-krb8z" Jul 08 17:25:16.532190 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.532193 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44a1876e-47a2-4c9c-ba62-a291294508ef-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-krb8z\" (UID: \"44a1876e-47a2-4c9c-ba62-a291294508ef\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-krb8z" Jul 08 17:25:16.532527 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.532510 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/44a1876e-47a2-4c9c-ba62-a291294508ef-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-krb8z\" (UID: \"44a1876e-47a2-4c9c-ba62-a291294508ef\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-krb8z" Jul 08 17:25:16.540768 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.540734 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5rdx\" (UniqueName: \"kubernetes.io/projected/44a1876e-47a2-4c9c-ba62-a291294508ef-kube-api-access-t5rdx\") pod \"cert-manager-operator-controller-manager-55c444bf86-krb8z\" (UID: \"44a1876e-47a2-4c9c-ba62-a291294508ef\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-krb8z" Jul 08 17:25:16.605121 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.605030 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-krb8z" Jul 08 17:25:16.745111 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:16.745076 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-krb8z"] Jul 08 17:25:16.749091 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:25:16.749062 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44a1876e_47a2_4c9c_ba62_a291294508ef.slice/crio-79b7bf9657d2178b6bd9d49e0da5ed2d55f8f40288b86dc1c20165835b04d498 WatchSource:0}: Error finding container 79b7bf9657d2178b6bd9d49e0da5ed2d55f8f40288b86dc1c20165835b04d498: Status 404 returned error can't find the container with id 79b7bf9657d2178b6bd9d49e0da5ed2d55f8f40288b86dc1c20165835b04d498 Jul 08 17:25:17.265184 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:17.265146 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-krb8z" event={"ID":"44a1876e-47a2-4c9c-ba62-a291294508ef","Type":"ContainerStarted","Data":"79b7bf9657d2178b6bd9d49e0da5ed2d55f8f40288b86dc1c20165835b04d498"} Jul 08 17:25:20.280709 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.280672 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-krb8z" event={"ID":"44a1876e-47a2-4c9c-ba62-a291294508ef","Type":"ContainerStarted","Data":"6115e2687da9140fa984e55ba1566bd1efcfe4871b24e1012fc356925aac7aba"} Jul 08 17:25:20.306361 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.306305 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-krb8z" podStartSLOduration=1.444799105 podStartE2EDuration="4.306289756s" totalImagesPullingTime="2.861490651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:25:20.303863748 +0000 UTC m=+554.529845862" watchObservedRunningTime="2026-07-08 17:25:20.306289756 +0000 UTC m=+554.532271875" Jul 08 17:25:20.684133 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.684096 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5tjtl"] Jul 08 17:25:20.686567 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.686550 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-5tjtl" Jul 08 17:25:20.689092 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.689058 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 17:25:20.689230 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.689127 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-6rrm7\"" Jul 08 17:25:20.689230 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.689190 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 17:25:20.698286 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.698258 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5tjtl"] Jul 08 17:25:20.769387 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.769352 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhwqf\" (UniqueName: \"kubernetes.io/projected/d3a4d6a2-e73a-4f27-a8ff-5e85a4385c68-kube-api-access-hhwqf\") pod \"cert-manager-webhook-6486ffd997-5tjtl\" (UID: \"d3a4d6a2-e73a-4f27-a8ff-5e85a4385c68\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5tjtl" Jul 08 17:25:20.769558 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.769398 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3a4d6a2-e73a-4f27-a8ff-5e85a4385c68-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5tjtl\" (UID: \"d3a4d6a2-e73a-4f27-a8ff-5e85a4385c68\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5tjtl" Jul 08 17:25:20.869896 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.869864 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhwqf\" (UniqueName: \"kubernetes.io/projected/d3a4d6a2-e73a-4f27-a8ff-5e85a4385c68-kube-api-access-hhwqf\") pod \"cert-manager-webhook-6486ffd997-5tjtl\" (UID: \"d3a4d6a2-e73a-4f27-a8ff-5e85a4385c68\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5tjtl" Jul 08 17:25:20.870101 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.869916 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3a4d6a2-e73a-4f27-a8ff-5e85a4385c68-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5tjtl\" (UID: \"d3a4d6a2-e73a-4f27-a8ff-5e85a4385c68\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5tjtl" Jul 08 17:25:20.918783 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.918746 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhwqf\" (UniqueName: \"kubernetes.io/projected/d3a4d6a2-e73a-4f27-a8ff-5e85a4385c68-kube-api-access-hhwqf\") pod \"cert-manager-webhook-6486ffd997-5tjtl\" (UID: \"d3a4d6a2-e73a-4f27-a8ff-5e85a4385c68\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5tjtl" Jul 08 17:25:20.918952 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.918879 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d3a4d6a2-e73a-4f27-a8ff-5e85a4385c68-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5tjtl\" (UID: \"d3a4d6a2-e73a-4f27-a8ff-5e85a4385c68\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5tjtl" Jul 08 17:25:20.979245 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.979165 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d"] Jul 08 17:25:20.981412 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.981396 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" Jul 08 17:25:20.983811 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.983790 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jhpsm\"" Jul 08 17:25:20.983941 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.983797 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 17:25:20.984031 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.984017 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 17:25:20.995773 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:20.995747 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d"] Jul 08 17:25:21.008480 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:21.008459 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-5tjtl" Jul 08 17:25:21.071258 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:21.071220 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/195f8411-4e4f-4357-940f-10558495f47f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d\" (UID: \"195f8411-4e4f-4357-940f-10558495f47f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" Jul 08 17:25:21.071429 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:21.071327 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfvsz\" (UniqueName: \"kubernetes.io/projected/195f8411-4e4f-4357-940f-10558495f47f-kube-api-access-lfvsz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d\" (UID: \"195f8411-4e4f-4357-940f-10558495f47f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" Jul 08 17:25:21.071429 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:21.071386 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/195f8411-4e4f-4357-940f-10558495f47f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d\" (UID: \"195f8411-4e4f-4357-940f-10558495f47f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" Jul 08 17:25:21.142505 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:21.142452 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5tjtl"] Jul 08 17:25:21.144678 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:25:21.144640 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3a4d6a2_e73a_4f27_a8ff_5e85a4385c68.slice/crio-1976cc3a9446a35fa8cefc7aad6c963f01c0de6c7471bf74865dc9d390aca2bb WatchSource:0}: Error finding container 1976cc3a9446a35fa8cefc7aad6c963f01c0de6c7471bf74865dc9d390aca2bb: Status 404 returned error can't find the container with id 1976cc3a9446a35fa8cefc7aad6c963f01c0de6c7471bf74865dc9d390aca2bb Jul 08 17:25:21.172500 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:21.172473 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfvsz\" (UniqueName: \"kubernetes.io/projected/195f8411-4e4f-4357-940f-10558495f47f-kube-api-access-lfvsz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d\" (UID: \"195f8411-4e4f-4357-940f-10558495f47f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" Jul 08 17:25:21.172665 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:21.172513 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/195f8411-4e4f-4357-940f-10558495f47f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d\" (UID: \"195f8411-4e4f-4357-940f-10558495f47f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" Jul 08 17:25:21.172665 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:21.172633 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/195f8411-4e4f-4357-940f-10558495f47f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d\" (UID: \"195f8411-4e4f-4357-940f-10558495f47f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" Jul 08 17:25:21.172898 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:21.172881 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/195f8411-4e4f-4357-940f-10558495f47f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d\" (UID: \"195f8411-4e4f-4357-940f-10558495f47f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" Jul 08 17:25:21.172936 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:21.172913 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/195f8411-4e4f-4357-940f-10558495f47f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d\" (UID: \"195f8411-4e4f-4357-940f-10558495f47f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" Jul 08 17:25:21.185570 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:21.185550 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfvsz\" (UniqueName: \"kubernetes.io/projected/195f8411-4e4f-4357-940f-10558495f47f-kube-api-access-lfvsz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d\" (UID: \"195f8411-4e4f-4357-940f-10558495f47f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" Jul 08 17:25:21.286113 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:21.286027 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-5tjtl" event={"ID":"d3a4d6a2-e73a-4f27-a8ff-5e85a4385c68","Type":"ContainerStarted","Data":"1976cc3a9446a35fa8cefc7aad6c963f01c0de6c7471bf74865dc9d390aca2bb"} Jul 08 17:25:21.291444 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:21.291424 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" Jul 08 17:25:21.426164 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:21.426135 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d"] Jul 08 17:25:21.429506 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:25:21.429474 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod195f8411_4e4f_4357_940f_10558495f47f.slice/crio-a76372ffa4c8b18ccacdfdec60f2d6a1286f63288d2993aaece067a5fc72a5ee WatchSource:0}: Error finding container a76372ffa4c8b18ccacdfdec60f2d6a1286f63288d2993aaece067a5fc72a5ee: Status 404 returned error can't find the container with id a76372ffa4c8b18ccacdfdec60f2d6a1286f63288d2993aaece067a5fc72a5ee Jul 08 17:25:22.292948 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:22.292909 2578 generic.go:356] "Generic (PLEG): container finished" podID="195f8411-4e4f-4357-940f-10558495f47f" containerID="04d4a730bec3e8ffdbed0174a7458b3f526e754925c070e99bcba263ba8db41f" exitCode=0 Jul 08 17:25:22.293379 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:22.292983 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" event={"ID":"195f8411-4e4f-4357-940f-10558495f47f","Type":"ContainerDied","Data":"04d4a730bec3e8ffdbed0174a7458b3f526e754925c070e99bcba263ba8db41f"} Jul 08 17:25:22.293379 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:22.293028 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" event={"ID":"195f8411-4e4f-4357-940f-10558495f47f","Type":"ContainerStarted","Data":"a76372ffa4c8b18ccacdfdec60f2d6a1286f63288d2993aaece067a5fc72a5ee"} Jul 08 17:25:24.304376 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:24.304334 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-5tjtl" event={"ID":"d3a4d6a2-e73a-4f27-a8ff-5e85a4385c68","Type":"ContainerStarted","Data":"c69ef26771180ea98f19db43c8a5f9a8d5e9cc74d95952d01f7e59b1ef7784e7"} Jul 08 17:25:24.304845 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:24.304466 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-5tjtl" Jul 08 17:25:24.323100 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:24.323059 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-5tjtl" podStartSLOduration=1.937214475 podStartE2EDuration="4.323045053s" totalImagesPullingTime="2.385830578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:25:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:25:24.32084218 +0000 UTC m=+558.546824300" watchObservedRunningTime="2026-07-08 17:25:24.323045053 +0000 UTC m=+558.549027172" Jul 08 17:25:25.312326 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:25.312288 2578 generic.go:356] "Generic (PLEG): container finished" podID="195f8411-4e4f-4357-940f-10558495f47f" containerID="9750e2890e29ecfe674031acd43ce033884073483c1236bb5e541c6fbe2c8c64" exitCode=0 Jul 08 17:25:25.312760 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:25.312370 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" event={"ID":"195f8411-4e4f-4357-940f-10558495f47f","Type":"ContainerDied","Data":"9750e2890e29ecfe674031acd43ce033884073483c1236bb5e541c6fbe2c8c64"} Jul 08 17:25:26.319954 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:26.319912 2578 generic.go:356] "Generic (PLEG): container finished" podID="195f8411-4e4f-4357-940f-10558495f47f" containerID="a34cf003729ec499b1efafa59a743e29853aeb80e0eb14f2bba71d72a24c0707" exitCode=0 Jul 08 17:25:26.320366 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:26.320014 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" event={"ID":"195f8411-4e4f-4357-940f-10558495f47f","Type":"ContainerDied","Data":"a34cf003729ec499b1efafa59a743e29853aeb80e0eb14f2bba71d72a24c0707"} Jul 08 17:25:27.455177 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:27.455142 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" Jul 08 17:25:27.631861 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:27.631775 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/195f8411-4e4f-4357-940f-10558495f47f-util\") pod \"195f8411-4e4f-4357-940f-10558495f47f\" (UID: \"195f8411-4e4f-4357-940f-10558495f47f\") " Jul 08 17:25:27.631861 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:27.631821 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/195f8411-4e4f-4357-940f-10558495f47f-bundle\") pod \"195f8411-4e4f-4357-940f-10558495f47f\" (UID: \"195f8411-4e4f-4357-940f-10558495f47f\") " Jul 08 17:25:27.632047 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:27.631876 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfvsz\" (UniqueName: \"kubernetes.io/projected/195f8411-4e4f-4357-940f-10558495f47f-kube-api-access-lfvsz\") pod \"195f8411-4e4f-4357-940f-10558495f47f\" (UID: \"195f8411-4e4f-4357-940f-10558495f47f\") " Jul 08 17:25:27.632286 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:27.632239 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/195f8411-4e4f-4357-940f-10558495f47f-bundle" (OuterVolumeSpecName: "bundle") pod "195f8411-4e4f-4357-940f-10558495f47f" (UID: "195f8411-4e4f-4357-940f-10558495f47f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:25:27.634131 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:27.634104 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/195f8411-4e4f-4357-940f-10558495f47f-kube-api-access-lfvsz" (OuterVolumeSpecName: "kube-api-access-lfvsz") pod "195f8411-4e4f-4357-940f-10558495f47f" (UID: "195f8411-4e4f-4357-940f-10558495f47f"). InnerVolumeSpecName "kube-api-access-lfvsz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:25:27.636394 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:27.636364 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/195f8411-4e4f-4357-940f-10558495f47f-util" (OuterVolumeSpecName: "util") pod "195f8411-4e4f-4357-940f-10558495f47f" (UID: "195f8411-4e4f-4357-940f-10558495f47f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:25:27.732541 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:27.732501 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/195f8411-4e4f-4357-940f-10558495f47f-util\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:25:27.732541 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:27.732532 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/195f8411-4e4f-4357-940f-10558495f47f-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:25:27.732541 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:27.732541 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfvsz\" (UniqueName: \"kubernetes.io/projected/195f8411-4e4f-4357-940f-10558495f47f-kube-api-access-lfvsz\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:25:28.329072 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:28.329041 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" Jul 08 17:25:28.329072 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:28.329063 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9k69d" event={"ID":"195f8411-4e4f-4357-940f-10558495f47f","Type":"ContainerDied","Data":"a76372ffa4c8b18ccacdfdec60f2d6a1286f63288d2993aaece067a5fc72a5ee"} Jul 08 17:25:28.329336 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:28.329110 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a76372ffa4c8b18ccacdfdec60f2d6a1286f63288d2993aaece067a5fc72a5ee" Jul 08 17:25:30.315488 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:30.315454 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-5tjtl" Jul 08 17:25:37.376460 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.376425 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjj5t"] Jul 08 17:25:37.376881 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.376728 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="195f8411-4e4f-4357-940f-10558495f47f" containerName="pull" Jul 08 17:25:37.376881 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.376740 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="195f8411-4e4f-4357-940f-10558495f47f" containerName="pull" Jul 08 17:25:37.376881 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.376748 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="195f8411-4e4f-4357-940f-10558495f47f" containerName="util" Jul 08 17:25:37.376881 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.376753 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="195f8411-4e4f-4357-940f-10558495f47f" containerName="util" Jul 08 17:25:37.376881 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.376764 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="195f8411-4e4f-4357-940f-10558495f47f" containerName="extract" Jul 08 17:25:37.376881 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.376769 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="195f8411-4e4f-4357-940f-10558495f47f" containerName="extract" Jul 08 17:25:37.376881 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.376817 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="195f8411-4e4f-4357-940f-10558495f47f" containerName="extract" Jul 08 17:25:37.419078 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.419041 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjj5t"] Jul 08 17:25:37.419235 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.419143 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjj5t" Jul 08 17:25:37.422001 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.421971 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 17:25:37.422831 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.422805 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-vkzx6\"" Jul 08 17:25:37.422966 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.422840 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 17:25:37.520773 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.520739 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fafa4837-ed67-4fcf-a39d-8979c76a7de3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wjj5t\" (UID: \"fafa4837-ed67-4fcf-a39d-8979c76a7de3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjj5t" Jul 08 17:25:37.520952 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.520792 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dgw9\" (UniqueName: \"kubernetes.io/projected/fafa4837-ed67-4fcf-a39d-8979c76a7de3-kube-api-access-2dgw9\") pod \"openshift-lws-operator-fd8ccff4c-wjj5t\" (UID: \"fafa4837-ed67-4fcf-a39d-8979c76a7de3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjj5t" Jul 08 17:25:37.621429 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.621389 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dgw9\" (UniqueName: \"kubernetes.io/projected/fafa4837-ed67-4fcf-a39d-8979c76a7de3-kube-api-access-2dgw9\") pod \"openshift-lws-operator-fd8ccff4c-wjj5t\" (UID: \"fafa4837-ed67-4fcf-a39d-8979c76a7de3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjj5t" Jul 08 17:25:37.621666 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.621465 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fafa4837-ed67-4fcf-a39d-8979c76a7de3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wjj5t\" (UID: \"fafa4837-ed67-4fcf-a39d-8979c76a7de3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjj5t" Jul 08 17:25:37.621814 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.621797 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fafa4837-ed67-4fcf-a39d-8979c76a7de3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wjj5t\" (UID: \"fafa4837-ed67-4fcf-a39d-8979c76a7de3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjj5t" Jul 08 17:25:37.630062 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.630012 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dgw9\" (UniqueName: \"kubernetes.io/projected/fafa4837-ed67-4fcf-a39d-8979c76a7de3-kube-api-access-2dgw9\") pod \"openshift-lws-operator-fd8ccff4c-wjj5t\" (UID: \"fafa4837-ed67-4fcf-a39d-8979c76a7de3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjj5t" Jul 08 17:25:37.728479 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.728444 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjj5t" Jul 08 17:25:37.880445 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:37.880393 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjj5t"] Jul 08 17:25:37.883048 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:25:37.883020 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfafa4837_ed67_4fcf_a39d_8979c76a7de3.slice/crio-a29903acd3402c3d2335e06d0de1e9457e33e30b5ed08d5b904797549ad33088 WatchSource:0}: Error finding container a29903acd3402c3d2335e06d0de1e9457e33e30b5ed08d5b904797549ad33088: Status 404 returned error can't find the container with id a29903acd3402c3d2335e06d0de1e9457e33e30b5ed08d5b904797549ad33088 Jul 08 17:25:38.372656 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:38.372618 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjj5t" event={"ID":"fafa4837-ed67-4fcf-a39d-8979c76a7de3","Type":"ContainerStarted","Data":"a29903acd3402c3d2335e06d0de1e9457e33e30b5ed08d5b904797549ad33088"} Jul 08 17:25:40.384527 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:40.384484 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjj5t" event={"ID":"fafa4837-ed67-4fcf-a39d-8979c76a7de3","Type":"ContainerStarted","Data":"d53fb150a2bf627967116e0eca63ca8a134cac525feca190f735a5a26299f53f"} Jul 08 17:25:40.403143 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:40.403087 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wjj5t" podStartSLOduration=1.563032775 podStartE2EDuration="3.403070678s" totalImagesPullingTime="1.840037903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:25:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:25:40.400940679 +0000 UTC m=+574.626922800" watchObservedRunningTime="2026-07-08 17:25:40.403070678 +0000 UTC m=+574.629052797" Jul 08 17:25:55.210723 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.210687 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q"] Jul 08 17:25:55.212873 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.212855 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" Jul 08 17:25:55.216643 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.216622 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 17:25:55.216760 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.216655 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 17:25:55.216760 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.216666 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 17:25:55.216760 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.216655 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lpcrq\"" Jul 08 17:25:55.225864 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.225843 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q"] Jul 08 17:25:55.255182 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.255153 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb2f85ac-9da4-495a-8790-e45fa0b59701-metrics-cert\") pod \"lws-controller-manager-8596cf9c79-4mr5q\" (UID: \"fb2f85ac-9da4-495a-8790-e45fa0b59701\") " pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" Jul 08 17:25:55.255327 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.255188 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb2f85ac-9da4-495a-8790-e45fa0b59701-cert\") pod \"lws-controller-manager-8596cf9c79-4mr5q\" (UID: \"fb2f85ac-9da4-495a-8790-e45fa0b59701\") " pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" Jul 08 17:25:55.255327 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.255213 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fb2f85ac-9da4-495a-8790-e45fa0b59701-manager-config\") pod \"lws-controller-manager-8596cf9c79-4mr5q\" (UID: \"fb2f85ac-9da4-495a-8790-e45fa0b59701\") " pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" Jul 08 17:25:55.255327 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.255272 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtqvd\" (UniqueName: \"kubernetes.io/projected/fb2f85ac-9da4-495a-8790-e45fa0b59701-kube-api-access-wtqvd\") pod \"lws-controller-manager-8596cf9c79-4mr5q\" (UID: \"fb2f85ac-9da4-495a-8790-e45fa0b59701\") " pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" Jul 08 17:25:55.355891 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.355852 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb2f85ac-9da4-495a-8790-e45fa0b59701-metrics-cert\") pod \"lws-controller-manager-8596cf9c79-4mr5q\" (UID: \"fb2f85ac-9da4-495a-8790-e45fa0b59701\") " pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" Jul 08 17:25:55.356077 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.355899 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb2f85ac-9da4-495a-8790-e45fa0b59701-cert\") pod \"lws-controller-manager-8596cf9c79-4mr5q\" (UID: \"fb2f85ac-9da4-495a-8790-e45fa0b59701\") " pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" Jul 08 17:25:55.356077 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.355938 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fb2f85ac-9da4-495a-8790-e45fa0b59701-manager-config\") pod \"lws-controller-manager-8596cf9c79-4mr5q\" (UID: \"fb2f85ac-9da4-495a-8790-e45fa0b59701\") " pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" Jul 08 17:25:55.356077 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.355961 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtqvd\" (UniqueName: \"kubernetes.io/projected/fb2f85ac-9da4-495a-8790-e45fa0b59701-kube-api-access-wtqvd\") pod \"lws-controller-manager-8596cf9c79-4mr5q\" (UID: \"fb2f85ac-9da4-495a-8790-e45fa0b59701\") " pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" Jul 08 17:25:55.356614 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.356571 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/fb2f85ac-9da4-495a-8790-e45fa0b59701-manager-config\") pod \"lws-controller-manager-8596cf9c79-4mr5q\" (UID: \"fb2f85ac-9da4-495a-8790-e45fa0b59701\") " pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" Jul 08 17:25:55.358278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.358260 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fb2f85ac-9da4-495a-8790-e45fa0b59701-cert\") pod \"lws-controller-manager-8596cf9c79-4mr5q\" (UID: \"fb2f85ac-9da4-495a-8790-e45fa0b59701\") " pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" Jul 08 17:25:55.358384 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.358367 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb2f85ac-9da4-495a-8790-e45fa0b59701-metrics-cert\") pod \"lws-controller-manager-8596cf9c79-4mr5q\" (UID: \"fb2f85ac-9da4-495a-8790-e45fa0b59701\") " pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" Jul 08 17:25:55.363971 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.363950 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtqvd\" (UniqueName: \"kubernetes.io/projected/fb2f85ac-9da4-495a-8790-e45fa0b59701-kube-api-access-wtqvd\") pod \"lws-controller-manager-8596cf9c79-4mr5q\" (UID: \"fb2f85ac-9da4-495a-8790-e45fa0b59701\") " pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" Jul 08 17:25:55.423108 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.423079 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k"] Jul 08 17:25:55.425635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.425620 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" Jul 08 17:25:55.428630 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.428606 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 17:25:55.428630 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.428623 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 17:25:55.428813 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.428801 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jhpsm\"" Jul 08 17:25:55.436489 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.436459 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k"] Jul 08 17:25:55.457078 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.457058 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3a47707-1deb-4f0f-8595-9bd93f216617-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k\" (UID: \"e3a47707-1deb-4f0f-8595-9bd93f216617\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" Jul 08 17:25:55.457189 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.457087 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-764wl\" (UniqueName: \"kubernetes.io/projected/e3a47707-1deb-4f0f-8595-9bd93f216617-kube-api-access-764wl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k\" (UID: \"e3a47707-1deb-4f0f-8595-9bd93f216617\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" Jul 08 17:25:55.457189 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.457122 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3a47707-1deb-4f0f-8595-9bd93f216617-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k\" (UID: \"e3a47707-1deb-4f0f-8595-9bd93f216617\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" Jul 08 17:25:55.522123 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.522047 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" Jul 08 17:25:55.558298 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.558268 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3a47707-1deb-4f0f-8595-9bd93f216617-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k\" (UID: \"e3a47707-1deb-4f0f-8595-9bd93f216617\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" Jul 08 17:25:55.558461 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.558306 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-764wl\" (UniqueName: \"kubernetes.io/projected/e3a47707-1deb-4f0f-8595-9bd93f216617-kube-api-access-764wl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k\" (UID: \"e3a47707-1deb-4f0f-8595-9bd93f216617\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" Jul 08 17:25:55.558461 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.558339 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3a47707-1deb-4f0f-8595-9bd93f216617-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k\" (UID: \"e3a47707-1deb-4f0f-8595-9bd93f216617\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" Jul 08 17:25:55.558659 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.558643 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3a47707-1deb-4f0f-8595-9bd93f216617-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k\" (UID: \"e3a47707-1deb-4f0f-8595-9bd93f216617\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" Jul 08 17:25:55.558727 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.558680 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3a47707-1deb-4f0f-8595-9bd93f216617-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k\" (UID: \"e3a47707-1deb-4f0f-8595-9bd93f216617\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" Jul 08 17:25:55.568833 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.568810 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-764wl\" (UniqueName: \"kubernetes.io/projected/e3a47707-1deb-4f0f-8595-9bd93f216617-kube-api-access-764wl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k\" (UID: \"e3a47707-1deb-4f0f-8595-9bd93f216617\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" Jul 08 17:25:55.649107 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.649079 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q"] Jul 08 17:25:55.652289 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:25:55.652266 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb2f85ac_9da4_495a_8790_e45fa0b59701.slice/crio-9c5e212eb01fc29e02d12f604fcfd336aeaf0613510e7f565ef4d6ca3b2981a6 WatchSource:0}: Error finding container 9c5e212eb01fc29e02d12f604fcfd336aeaf0613510e7f565ef4d6ca3b2981a6: Status 404 returned error can't find the container with id 9c5e212eb01fc29e02d12f604fcfd336aeaf0613510e7f565ef4d6ca3b2981a6 Jul 08 17:25:55.734517 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.734495 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" Jul 08 17:25:55.873177 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:55.873150 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k"] Jul 08 17:25:55.875355 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:25:55.875326 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3a47707_1deb_4f0f_8595_9bd93f216617.slice/crio-129e692848697bfa48b60fd902deec830fe461046e424cbf5e482724e4567f0b WatchSource:0}: Error finding container 129e692848697bfa48b60fd902deec830fe461046e424cbf5e482724e4567f0b: Status 404 returned error can't find the container with id 129e692848697bfa48b60fd902deec830fe461046e424cbf5e482724e4567f0b Jul 08 17:25:56.450660 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:56.450613 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" event={"ID":"fb2f85ac-9da4-495a-8790-e45fa0b59701","Type":"ContainerStarted","Data":"9c5e212eb01fc29e02d12f604fcfd336aeaf0613510e7f565ef4d6ca3b2981a6"} Jul 08 17:25:56.452266 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:56.452237 2578 generic.go:356] "Generic (PLEG): container finished" podID="e3a47707-1deb-4f0f-8595-9bd93f216617" containerID="74418db172425f8a480b4b8268e191d280d6da778773dfa5eaaf3a9f9a72035c" exitCode=0 Jul 08 17:25:56.452391 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:56.452344 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" event={"ID":"e3a47707-1deb-4f0f-8595-9bd93f216617","Type":"ContainerDied","Data":"74418db172425f8a480b4b8268e191d280d6da778773dfa5eaaf3a9f9a72035c"} Jul 08 17:25:56.452434 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:56.452394 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" event={"ID":"e3a47707-1deb-4f0f-8595-9bd93f216617","Type":"ContainerStarted","Data":"129e692848697bfa48b60fd902deec830fe461046e424cbf5e482724e4567f0b"} Jul 08 17:25:57.458085 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:57.458055 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" event={"ID":"fb2f85ac-9da4-495a-8790-e45fa0b59701","Type":"ContainerStarted","Data":"08cfa8a34332727cf6ad6e2aa1f84e4897120c7df6eeb5cfe770aad19eed7120"} Jul 08 17:25:57.458468 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:57.458152 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" Jul 08 17:25:57.460163 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:57.460128 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" event={"ID":"e3a47707-1deb-4f0f-8595-9bd93f216617","Type":"ContainerStarted","Data":"813456576b7a53dd9192f4341f7a61909b5e739ce2f36025ae83d974b0ac4d44"} Jul 08 17:25:57.475984 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:57.475928 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" podStartSLOduration=0.894652259 podStartE2EDuration="2.475908727s" totalImagesPullingTime="1.581256468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:25:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:25:57.473681377 +0000 UTC m=+591.699663517" watchObservedRunningTime="2026-07-08 17:25:57.475908727 +0000 UTC m=+591.701890850" Jul 08 17:25:58.465403 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:58.465369 2578 generic.go:356] "Generic (PLEG): container finished" podID="e3a47707-1deb-4f0f-8595-9bd93f216617" containerID="813456576b7a53dd9192f4341f7a61909b5e739ce2f36025ae83d974b0ac4d44" exitCode=0 Jul 08 17:25:58.465819 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:58.465452 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" event={"ID":"e3a47707-1deb-4f0f-8595-9bd93f216617","Type":"ContainerDied","Data":"813456576b7a53dd9192f4341f7a61909b5e739ce2f36025ae83d974b0ac4d44"} Jul 08 17:25:59.472611 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:59.472563 2578 generic.go:356] "Generic (PLEG): container finished" podID="e3a47707-1deb-4f0f-8595-9bd93f216617" containerID="5287057a6e1b8da93735e169ba995c009750355ced2b961af5c368aa8b6a44c5" exitCode=0 Jul 08 17:25:59.472988 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:25:59.472648 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" event={"ID":"e3a47707-1deb-4f0f-8595-9bd93f216617","Type":"ContainerDied","Data":"5287057a6e1b8da93735e169ba995c009750355ced2b961af5c368aa8b6a44c5"} Jul 08 17:26:00.604294 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:00.604272 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" Jul 08 17:26:00.705019 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:00.704989 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-764wl\" (UniqueName: \"kubernetes.io/projected/e3a47707-1deb-4f0f-8595-9bd93f216617-kube-api-access-764wl\") pod \"e3a47707-1deb-4f0f-8595-9bd93f216617\" (UID: \"e3a47707-1deb-4f0f-8595-9bd93f216617\") " Jul 08 17:26:00.705193 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:00.705040 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3a47707-1deb-4f0f-8595-9bd93f216617-util\") pod \"e3a47707-1deb-4f0f-8595-9bd93f216617\" (UID: \"e3a47707-1deb-4f0f-8595-9bd93f216617\") " Jul 08 17:26:00.705193 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:00.705070 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3a47707-1deb-4f0f-8595-9bd93f216617-bundle\") pod \"e3a47707-1deb-4f0f-8595-9bd93f216617\" (UID: \"e3a47707-1deb-4f0f-8595-9bd93f216617\") " Jul 08 17:26:00.705789 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:00.705760 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3a47707-1deb-4f0f-8595-9bd93f216617-bundle" (OuterVolumeSpecName: "bundle") pod "e3a47707-1deb-4f0f-8595-9bd93f216617" (UID: "e3a47707-1deb-4f0f-8595-9bd93f216617"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:26:00.707104 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:00.707079 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3a47707-1deb-4f0f-8595-9bd93f216617-kube-api-access-764wl" (OuterVolumeSpecName: "kube-api-access-764wl") pod "e3a47707-1deb-4f0f-8595-9bd93f216617" (UID: "e3a47707-1deb-4f0f-8595-9bd93f216617"). InnerVolumeSpecName "kube-api-access-764wl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:26:00.712714 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:00.712671 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3a47707-1deb-4f0f-8595-9bd93f216617-util" (OuterVolumeSpecName: "util") pod "e3a47707-1deb-4f0f-8595-9bd93f216617" (UID: "e3a47707-1deb-4f0f-8595-9bd93f216617"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:26:00.805613 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:00.805528 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e3a47707-1deb-4f0f-8595-9bd93f216617-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:26:00.805613 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:00.805553 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-764wl\" (UniqueName: \"kubernetes.io/projected/e3a47707-1deb-4f0f-8595-9bd93f216617-kube-api-access-764wl\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:26:00.805613 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:00.805564 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e3a47707-1deb-4f0f-8595-9bd93f216617-util\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:26:01.488706 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:01.488657 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" event={"ID":"e3a47707-1deb-4f0f-8595-9bd93f216617","Type":"ContainerDied","Data":"129e692848697bfa48b60fd902deec830fe461046e424cbf5e482724e4567f0b"} Jul 08 17:26:01.488915 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:01.488793 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="129e692848697bfa48b60fd902deec830fe461046e424cbf5e482724e4567f0b" Jul 08 17:26:01.488915 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:01.488726 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afjd48k" Jul 08 17:26:08.467560 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:08.467528 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8596cf9c79-4mr5q" Jul 08 17:26:11.843964 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.843929 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5"] Jul 08 17:26:11.844408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.844243 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3a47707-1deb-4f0f-8595-9bd93f216617" containerName="pull" Jul 08 17:26:11.844408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.844255 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3a47707-1deb-4f0f-8595-9bd93f216617" containerName="pull" Jul 08 17:26:11.844408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.844270 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3a47707-1deb-4f0f-8595-9bd93f216617" containerName="util" Jul 08 17:26:11.844408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.844275 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3a47707-1deb-4f0f-8595-9bd93f216617" containerName="util" Jul 08 17:26:11.844408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.844285 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3a47707-1deb-4f0f-8595-9bd93f216617" containerName="extract" Jul 08 17:26:11.844408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.844290 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3a47707-1deb-4f0f-8595-9bd93f216617" containerName="extract" Jul 08 17:26:11.844408 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.844337 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3a47707-1deb-4f0f-8595-9bd93f216617" containerName="extract" Jul 08 17:26:11.847484 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.847465 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:11.854617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.854596 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-748sr\"" Jul 08 17:26:11.854786 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.854768 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 17:26:11.855282 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.855269 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 17:26:11.855369 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.855346 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 17:26:11.855723 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.855705 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 17:26:11.867318 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.867295 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5"] Jul 08 17:26:11.890101 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.890077 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c051e1c-f006-4118-a7cd-6825f368f632-webhook-cert\") pod \"opendatahub-operator-controller-manager-564755f94b-2djl5\" (UID: \"2c051e1c-f006-4118-a7cd-6825f368f632\") " pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:11.890207 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.890110 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c051e1c-f006-4118-a7cd-6825f368f632-tmp\") pod \"opendatahub-operator-controller-manager-564755f94b-2djl5\" (UID: \"2c051e1c-f006-4118-a7cd-6825f368f632\") " pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:11.890207 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.890140 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2c051e1c-f006-4118-a7cd-6825f368f632-manifests\") pod \"opendatahub-operator-controller-manager-564755f94b-2djl5\" (UID: \"2c051e1c-f006-4118-a7cd-6825f368f632\") " pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:11.890207 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.890198 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf67b\" (UniqueName: \"kubernetes.io/projected/2c051e1c-f006-4118-a7cd-6825f368f632-kube-api-access-zf67b\") pod \"opendatahub-operator-controller-manager-564755f94b-2djl5\" (UID: \"2c051e1c-f006-4118-a7cd-6825f368f632\") " pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:11.890309 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.890243 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2c051e1c-f006-4118-a7cd-6825f368f632-apiservice-cert\") pod \"opendatahub-operator-controller-manager-564755f94b-2djl5\" (UID: \"2c051e1c-f006-4118-a7cd-6825f368f632\") " pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:11.991375 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.991343 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2c051e1c-f006-4118-a7cd-6825f368f632-apiservice-cert\") pod \"opendatahub-operator-controller-manager-564755f94b-2djl5\" (UID: \"2c051e1c-f006-4118-a7cd-6825f368f632\") " pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:11.991496 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.991399 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c051e1c-f006-4118-a7cd-6825f368f632-webhook-cert\") pod \"opendatahub-operator-controller-manager-564755f94b-2djl5\" (UID: \"2c051e1c-f006-4118-a7cd-6825f368f632\") " pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:11.991496 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.991430 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c051e1c-f006-4118-a7cd-6825f368f632-tmp\") pod \"opendatahub-operator-controller-manager-564755f94b-2djl5\" (UID: \"2c051e1c-f006-4118-a7cd-6825f368f632\") " pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:11.991496 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.991451 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2c051e1c-f006-4118-a7cd-6825f368f632-manifests\") pod \"opendatahub-operator-controller-manager-564755f94b-2djl5\" (UID: \"2c051e1c-f006-4118-a7cd-6825f368f632\") " pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:11.991638 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.991495 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf67b\" (UniqueName: \"kubernetes.io/projected/2c051e1c-f006-4118-a7cd-6825f368f632-kube-api-access-zf67b\") pod \"opendatahub-operator-controller-manager-564755f94b-2djl5\" (UID: \"2c051e1c-f006-4118-a7cd-6825f368f632\") " pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:11.991943 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.991918 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2c051e1c-f006-4118-a7cd-6825f368f632-manifests\") pod \"opendatahub-operator-controller-manager-564755f94b-2djl5\" (UID: \"2c051e1c-f006-4118-a7cd-6825f368f632\") " pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:11.993617 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.993576 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c051e1c-f006-4118-a7cd-6825f368f632-tmp\") pod \"opendatahub-operator-controller-manager-564755f94b-2djl5\" (UID: \"2c051e1c-f006-4118-a7cd-6825f368f632\") " pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:11.993752 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.993732 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2c051e1c-f006-4118-a7cd-6825f368f632-apiservice-cert\") pod \"opendatahub-operator-controller-manager-564755f94b-2djl5\" (UID: \"2c051e1c-f006-4118-a7cd-6825f368f632\") " pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:11.993953 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.993932 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c051e1c-f006-4118-a7cd-6825f368f632-webhook-cert\") pod \"opendatahub-operator-controller-manager-564755f94b-2djl5\" (UID: \"2c051e1c-f006-4118-a7cd-6825f368f632\") " pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:11.999500 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:11.999483 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf67b\" (UniqueName: \"kubernetes.io/projected/2c051e1c-f006-4118-a7cd-6825f368f632-kube-api-access-zf67b\") pod \"opendatahub-operator-controller-manager-564755f94b-2djl5\" (UID: \"2c051e1c-f006-4118-a7cd-6825f368f632\") " pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:12.157275 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:12.157191 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:12.286295 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:12.286271 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5"] Jul 08 17:26:12.289292 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:26:12.289268 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c051e1c_f006_4118_a7cd_6825f368f632.slice/crio-5249c9a4f8fb5d542575764af0d7d9982e5db76a1c581d7dddc209be1e6415ab WatchSource:0}: Error finding container 5249c9a4f8fb5d542575764af0d7d9982e5db76a1c581d7dddc209be1e6415ab: Status 404 returned error can't find the container with id 5249c9a4f8fb5d542575764af0d7d9982e5db76a1c581d7dddc209be1e6415ab Jul 08 17:26:12.291059 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:12.291038 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 17:26:12.546759 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:12.546721 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" event={"ID":"2c051e1c-f006-4118-a7cd-6825f368f632","Type":"ContainerStarted","Data":"5249c9a4f8fb5d542575764af0d7d9982e5db76a1c581d7dddc209be1e6415ab"} Jul 08 17:26:15.562212 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:15.562166 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" event={"ID":"2c051e1c-f006-4118-a7cd-6825f368f632","Type":"ContainerStarted","Data":"fe59f9443650fe547889f712f610cd5565ad582f98c88742e5749bfbab933e09"} Jul 08 17:26:16.568153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:16.568063 2578 generic.go:356] "Generic (PLEG): container finished" podID="2c051e1c-f006-4118-a7cd-6825f368f632" containerID="fe59f9443650fe547889f712f610cd5565ad582f98c88742e5749bfbab933e09" exitCode=0 Jul 08 17:26:16.568153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:16.568111 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" event={"ID":"2c051e1c-f006-4118-a7cd-6825f368f632","Type":"ContainerDied","Data":"fe59f9443650fe547889f712f610cd5565ad582f98c88742e5749bfbab933e09"} Jul 08 17:26:17.575882 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:17.575845 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" event={"ID":"2c051e1c-f006-4118-a7cd-6825f368f632","Type":"ContainerStarted","Data":"51180a3f34addafbf86cb89291fa5e6ea087aa63c1db9f60d3dff17b2ee54538"} Jul 08 17:26:17.576376 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:17.576037 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:17.599648 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:17.599595 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" podStartSLOduration=2.277654759 podStartE2EDuration="6.599564201s" totalImagesPullingTime="3.24405379s" totalInitContainerRuntime="1.077855652s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:26:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:26:17.597057162 +0000 UTC m=+611.823039281" watchObservedRunningTime="2026-07-08 17:26:17.599564201 +0000 UTC m=+611.825546320" Jul 08 17:26:28.582621 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:28.582571 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-564755f94b-2djl5" Jul 08 17:26:36.745925 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:36.745888 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6d86b88db9-rvndc"] Jul 08 17:26:36.751453 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:36.751423 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6d86b88db9-rvndc" Jul 08 17:26:36.753847 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:36.753821 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6qz6r\"" Jul 08 17:26:36.754091 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:36.753877 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 17:26:36.754171 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:36.753964 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 17:26:36.763790 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:36.763767 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6d86b88db9-rvndc"] Jul 08 17:26:36.809885 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:36.809833 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f-tls-certs\") pod \"kube-auth-proxy-6d86b88db9-rvndc\" (UID: \"7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f\") " pod="openshift-ingress/kube-auth-proxy-6d86b88db9-rvndc" Jul 08 17:26:36.810027 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:36.809898 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kfxq\" (UniqueName: \"kubernetes.io/projected/7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f-kube-api-access-8kfxq\") pod \"kube-auth-proxy-6d86b88db9-rvndc\" (UID: \"7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f\") " pod="openshift-ingress/kube-auth-proxy-6d86b88db9-rvndc" Jul 08 17:26:36.810027 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:36.809932 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f-tmp\") pod \"kube-auth-proxy-6d86b88db9-rvndc\" (UID: \"7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f\") " pod="openshift-ingress/kube-auth-proxy-6d86b88db9-rvndc" Jul 08 17:26:36.910965 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:36.910928 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f-tls-certs\") pod \"kube-auth-proxy-6d86b88db9-rvndc\" (UID: \"7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f\") " pod="openshift-ingress/kube-auth-proxy-6d86b88db9-rvndc" Jul 08 17:26:36.911140 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:36.910983 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kfxq\" (UniqueName: \"kubernetes.io/projected/7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f-kube-api-access-8kfxq\") pod \"kube-auth-proxy-6d86b88db9-rvndc\" (UID: \"7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f\") " pod="openshift-ingress/kube-auth-proxy-6d86b88db9-rvndc" Jul 08 17:26:36.911140 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:36.911017 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f-tmp\") pod \"kube-auth-proxy-6d86b88db9-rvndc\" (UID: \"7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f\") " pod="openshift-ingress/kube-auth-proxy-6d86b88db9-rvndc" Jul 08 17:26:36.913450 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:36.913419 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f-tmp\") pod \"kube-auth-proxy-6d86b88db9-rvndc\" (UID: \"7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f\") " pod="openshift-ingress/kube-auth-proxy-6d86b88db9-rvndc" Jul 08 17:26:36.913561 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:36.913491 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f-tls-certs\") pod \"kube-auth-proxy-6d86b88db9-rvndc\" (UID: \"7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f\") " pod="openshift-ingress/kube-auth-proxy-6d86b88db9-rvndc" Jul 08 17:26:36.919494 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:36.919473 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kfxq\" (UniqueName: \"kubernetes.io/projected/7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f-kube-api-access-8kfxq\") pod \"kube-auth-proxy-6d86b88db9-rvndc\" (UID: \"7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f\") " pod="openshift-ingress/kube-auth-proxy-6d86b88db9-rvndc" Jul 08 17:26:37.066172 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:37.066081 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6d86b88db9-rvndc" Jul 08 17:26:37.213082 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:37.213055 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6d86b88db9-rvndc"] Jul 08 17:26:37.215942 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:26:37.215901 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a372ab1_23fa_4c4e_ae1b_fd5431f18f6f.slice/crio-25b7ae8ce619cfecb9e2e11ae90ffa8cf01eaa2bb502806654228c3f893ebc71 WatchSource:0}: Error finding container 25b7ae8ce619cfecb9e2e11ae90ffa8cf01eaa2bb502806654228c3f893ebc71: Status 404 returned error can't find the container with id 25b7ae8ce619cfecb9e2e11ae90ffa8cf01eaa2bb502806654228c3f893ebc71 Jul 08 17:26:37.668598 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:37.668546 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6d86b88db9-rvndc" event={"ID":"7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f","Type":"ContainerStarted","Data":"25b7ae8ce619cfecb9e2e11ae90ffa8cf01eaa2bb502806654228c3f893ebc71"} Jul 08 17:26:40.686062 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:40.686027 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6d86b88db9-rvndc" event={"ID":"7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f","Type":"ContainerStarted","Data":"31be2ad714ba998d7bf4e2cb9849e46b717264abfb2572464889bdc2b7a9d179"} Jul 08 17:26:40.707548 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:26:40.707489 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6d86b88db9-rvndc" podStartSLOduration=1.839845012 podStartE2EDuration="4.707469457s" totalImagesPullingTime="2.867624445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:26:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:26:40.705472049 +0000 UTC m=+634.931454170" watchObservedRunningTime="2026-07-08 17:26:40.707469457 +0000 UTC m=+634.933451579" Jul 08 17:27:48.103965 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.103933 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq"] Jul 08 17:27:48.107291 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.107276 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" Jul 08 17:27:48.109917 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.109895 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 17:27:48.109917 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.109896 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cncpd\"" Jul 08 17:27:48.110673 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.110659 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 17:27:48.115668 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.115648 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq"] Jul 08 17:27:48.160389 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.160362 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p945\" (UniqueName: \"kubernetes.io/projected/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-kube-api-access-9p945\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq\" (UID: \"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" Jul 08 17:27:48.160492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.160422 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq\" (UID: \"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" Jul 08 17:27:48.160492 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.160472 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq\" (UID: \"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" Jul 08 17:27:48.261106 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.261082 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq\" (UID: \"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" Jul 08 17:27:48.261206 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.261128 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9p945\" (UniqueName: \"kubernetes.io/projected/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-kube-api-access-9p945\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq\" (UID: \"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" Jul 08 17:27:48.261206 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.261172 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq\" (UID: \"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" Jul 08 17:27:48.261416 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.261399 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq\" (UID: \"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" Jul 08 17:27:48.261496 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.261478 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq\" (UID: \"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" Jul 08 17:27:48.269763 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.269748 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p945\" (UniqueName: \"kubernetes.io/projected/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-kube-api-access-9p945\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq\" (UID: \"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" Jul 08 17:27:48.417253 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.417204 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" Jul 08 17:27:48.504542 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.504513 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl"] Jul 08 17:27:48.508346 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.508328 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" Jul 08 17:27:48.517187 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.517096 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl"] Jul 08 17:27:48.546930 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.546907 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq"] Jul 08 17:27:48.549270 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:27:48.549242 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7052a54_e1c0_4dd4_b8f1_ffa9a6c33ccb.slice/crio-fcf6ffd0ef8ee0062c9318feda32a14f5b0dbdd7230a4c116133c4bb9905ae1c WatchSource:0}: Error finding container fcf6ffd0ef8ee0062c9318feda32a14f5b0dbdd7230a4c116133c4bb9905ae1c: Status 404 returned error can't find the container with id fcf6ffd0ef8ee0062c9318feda32a14f5b0dbdd7230a4c116133c4bb9905ae1c Jul 08 17:27:48.563982 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.563958 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm958\" (UniqueName: \"kubernetes.io/projected/ae794944-014d-49fc-84e4-4af22f676baf-kube-api-access-bm958\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl\" (UID: \"ae794944-014d-49fc-84e4-4af22f676baf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" Jul 08 17:27:48.564088 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.564009 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae794944-014d-49fc-84e4-4af22f676baf-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl\" (UID: \"ae794944-014d-49fc-84e4-4af22f676baf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" Jul 08 17:27:48.564088 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.564059 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae794944-014d-49fc-84e4-4af22f676baf-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl\" (UID: \"ae794944-014d-49fc-84e4-4af22f676baf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" Jul 08 17:27:48.665398 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.665362 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm958\" (UniqueName: \"kubernetes.io/projected/ae794944-014d-49fc-84e4-4af22f676baf-kube-api-access-bm958\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl\" (UID: \"ae794944-014d-49fc-84e4-4af22f676baf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" Jul 08 17:27:48.665528 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.665421 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae794944-014d-49fc-84e4-4af22f676baf-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl\" (UID: \"ae794944-014d-49fc-84e4-4af22f676baf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" Jul 08 17:27:48.665528 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.665462 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae794944-014d-49fc-84e4-4af22f676baf-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl\" (UID: \"ae794944-014d-49fc-84e4-4af22f676baf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" Jul 08 17:27:48.665867 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.665846 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae794944-014d-49fc-84e4-4af22f676baf-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl\" (UID: \"ae794944-014d-49fc-84e4-4af22f676baf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" Jul 08 17:27:48.665867 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.665859 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae794944-014d-49fc-84e4-4af22f676baf-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl\" (UID: \"ae794944-014d-49fc-84e4-4af22f676baf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" Jul 08 17:27:48.673768 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.673725 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm958\" (UniqueName: \"kubernetes.io/projected/ae794944-014d-49fc-84e4-4af22f676baf-kube-api-access-bm958\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl\" (UID: \"ae794944-014d-49fc-84e4-4af22f676baf\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" Jul 08 17:27:48.819024 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.818990 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" Jul 08 17:27:48.900446 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.900414 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t"] Jul 08 17:27:48.904757 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.904734 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" Jul 08 17:27:48.911532 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.911509 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t"] Jul 08 17:27:48.942562 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.942422 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl"] Jul 08 17:27:48.944980 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:27:48.944954 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae794944_014d_49fc_84e4_4af22f676baf.slice/crio-b1347c9db0eb07d029d72be268ba3eb758ce523106f29bc6f34d07352337b91d WatchSource:0}: Error finding container b1347c9db0eb07d029d72be268ba3eb758ce523106f29bc6f34d07352337b91d: Status 404 returned error can't find the container with id b1347c9db0eb07d029d72be268ba3eb758ce523106f29bc6f34d07352337b91d Jul 08 17:27:48.967310 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.967289 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t\" (UID: \"3f041301-1c23-44be-ab8e-1cd9d6f0f62a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" Jul 08 17:27:48.967430 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.967413 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t\" (UID: \"3f041301-1c23-44be-ab8e-1cd9d6f0f62a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" Jul 08 17:27:48.967489 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.967465 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mkln\" (UniqueName: \"kubernetes.io/projected/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-kube-api-access-9mkln\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t\" (UID: \"3f041301-1c23-44be-ab8e-1cd9d6f0f62a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" Jul 08 17:27:48.990819 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.990794 2578 generic.go:356] "Generic (PLEG): container finished" podID="d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb" containerID="2af81a4886311c14167ec83e704c6850be705f8959c8e9fa87ba72480f8aec7c" exitCode=0 Jul 08 17:27:48.990897 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.990874 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" event={"ID":"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb","Type":"ContainerDied","Data":"2af81a4886311c14167ec83e704c6850be705f8959c8e9fa87ba72480f8aec7c"} Jul 08 17:27:48.990953 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.990915 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" event={"ID":"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb","Type":"ContainerStarted","Data":"fcf6ffd0ef8ee0062c9318feda32a14f5b0dbdd7230a4c116133c4bb9905ae1c"} Jul 08 17:27:48.992223 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:48.992201 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" event={"ID":"ae794944-014d-49fc-84e4-4af22f676baf","Type":"ContainerStarted","Data":"b1347c9db0eb07d029d72be268ba3eb758ce523106f29bc6f34d07352337b91d"} Jul 08 17:27:49.068490 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.068461 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t\" (UID: \"3f041301-1c23-44be-ab8e-1cd9d6f0f62a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" Jul 08 17:27:49.068628 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.068501 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mkln\" (UniqueName: \"kubernetes.io/projected/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-kube-api-access-9mkln\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t\" (UID: \"3f041301-1c23-44be-ab8e-1cd9d6f0f62a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" Jul 08 17:27:49.068628 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.068525 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t\" (UID: \"3f041301-1c23-44be-ab8e-1cd9d6f0f62a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" Jul 08 17:27:49.068875 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.068858 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t\" (UID: \"3f041301-1c23-44be-ab8e-1cd9d6f0f62a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" Jul 08 17:27:49.068954 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.068879 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t\" (UID: \"3f041301-1c23-44be-ab8e-1cd9d6f0f62a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" Jul 08 17:27:49.077194 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.077177 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mkln\" (UniqueName: \"kubernetes.io/projected/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-kube-api-access-9mkln\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t\" (UID: \"3f041301-1c23-44be-ab8e-1cd9d6f0f62a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" Jul 08 17:27:49.215637 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.215543 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" Jul 08 17:27:49.298860 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.298836 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h"] Jul 08 17:27:49.302318 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.302296 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" Jul 08 17:27:49.313302 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.313195 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h"] Jul 08 17:27:49.355348 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.355294 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t"] Jul 08 17:27:49.357364 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:27:49.357336 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f041301_1c23_44be_ab8e_1cd9d6f0f62a.slice/crio-220a91642c0994c10627078635eb57e266c9bd265deaeb172973daace7a7647e WatchSource:0}: Error finding container 220a91642c0994c10627078635eb57e266c9bd265deaeb172973daace7a7647e: Status 404 returned error can't find the container with id 220a91642c0994c10627078635eb57e266c9bd265deaeb172973daace7a7647e Jul 08 17:27:49.370839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.370816 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27a036b2-a643-4d1b-a38d-9b86bd249d71-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h\" (UID: \"27a036b2-a643-4d1b-a38d-9b86bd249d71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" Jul 08 17:27:49.370942 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.370859 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nt2k\" (UniqueName: \"kubernetes.io/projected/27a036b2-a643-4d1b-a38d-9b86bd249d71-kube-api-access-5nt2k\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h\" (UID: \"27a036b2-a643-4d1b-a38d-9b86bd249d71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" Jul 08 17:27:49.370942 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.370915 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27a036b2-a643-4d1b-a38d-9b86bd249d71-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h\" (UID: \"27a036b2-a643-4d1b-a38d-9b86bd249d71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" Jul 08 17:27:49.471862 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.471813 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27a036b2-a643-4d1b-a38d-9b86bd249d71-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h\" (UID: \"27a036b2-a643-4d1b-a38d-9b86bd249d71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" Jul 08 17:27:49.471862 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.471859 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nt2k\" (UniqueName: \"kubernetes.io/projected/27a036b2-a643-4d1b-a38d-9b86bd249d71-kube-api-access-5nt2k\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h\" (UID: \"27a036b2-a643-4d1b-a38d-9b86bd249d71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" Jul 08 17:27:49.472010 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.471888 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27a036b2-a643-4d1b-a38d-9b86bd249d71-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h\" (UID: \"27a036b2-a643-4d1b-a38d-9b86bd249d71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" Jul 08 17:27:49.472143 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.472124 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27a036b2-a643-4d1b-a38d-9b86bd249d71-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h\" (UID: \"27a036b2-a643-4d1b-a38d-9b86bd249d71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" Jul 08 17:27:49.472203 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.472153 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27a036b2-a643-4d1b-a38d-9b86bd249d71-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h\" (UID: \"27a036b2-a643-4d1b-a38d-9b86bd249d71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" Jul 08 17:27:49.479944 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.479922 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nt2k\" (UniqueName: \"kubernetes.io/projected/27a036b2-a643-4d1b-a38d-9b86bd249d71-kube-api-access-5nt2k\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h\" (UID: \"27a036b2-a643-4d1b-a38d-9b86bd249d71\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" Jul 08 17:27:49.614780 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.614744 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" Jul 08 17:27:49.761234 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.761208 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h"] Jul 08 17:27:49.763187 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:27:49.763163 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27a036b2_a643_4d1b_a38d_9b86bd249d71.slice/crio-87adef69cd4b67bf57f91e750ba06e8a01164cdc302587404ac20e44e21d528a WatchSource:0}: Error finding container 87adef69cd4b67bf57f91e750ba06e8a01164cdc302587404ac20e44e21d528a: Status 404 returned error can't find the container with id 87adef69cd4b67bf57f91e750ba06e8a01164cdc302587404ac20e44e21d528a Jul 08 17:27:49.998681 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.998610 2578 generic.go:356] "Generic (PLEG): container finished" podID="ae794944-014d-49fc-84e4-4af22f676baf" containerID="c828e4c27465bd0e6cd59bae3a08e8933c249c40e8c2fc496a2c9d7affdd5140" exitCode=0 Jul 08 17:27:49.998681 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:49.998656 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" event={"ID":"ae794944-014d-49fc-84e4-4af22f676baf","Type":"ContainerDied","Data":"c828e4c27465bd0e6cd59bae3a08e8933c249c40e8c2fc496a2c9d7affdd5140"} Jul 08 17:27:50.000222 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:50.000198 2578 generic.go:356] "Generic (PLEG): container finished" podID="27a036b2-a643-4d1b-a38d-9b86bd249d71" containerID="73ecee95a4037dbf3df68c692f6f4eaa535422c516490898b85219e75d763144" exitCode=0 Jul 08 17:27:50.000318 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:50.000295 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" event={"ID":"27a036b2-a643-4d1b-a38d-9b86bd249d71","Type":"ContainerDied","Data":"73ecee95a4037dbf3df68c692f6f4eaa535422c516490898b85219e75d763144"} Jul 08 17:27:50.000375 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:50.000349 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" event={"ID":"27a036b2-a643-4d1b-a38d-9b86bd249d71","Type":"ContainerStarted","Data":"87adef69cd4b67bf57f91e750ba06e8a01164cdc302587404ac20e44e21d528a"} Jul 08 17:27:50.002240 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:50.002202 2578 generic.go:356] "Generic (PLEG): container finished" podID="d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb" containerID="be8eb6dc18ae1a5c109c7184e30cfd0f7d2d7d9e11a614c0ca3c70db645c443e" exitCode=0 Jul 08 17:27:50.002325 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:50.002285 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" event={"ID":"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb","Type":"ContainerDied","Data":"be8eb6dc18ae1a5c109c7184e30cfd0f7d2d7d9e11a614c0ca3c70db645c443e"} Jul 08 17:27:50.003831 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:50.003812 2578 generic.go:356] "Generic (PLEG): container finished" podID="3f041301-1c23-44be-ab8e-1cd9d6f0f62a" containerID="9ed3516b41f58127e294da380f02a069216b8ebfd5f998569de1b586b1314a8b" exitCode=0 Jul 08 17:27:50.003925 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:50.003839 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" event={"ID":"3f041301-1c23-44be-ab8e-1cd9d6f0f62a","Type":"ContainerDied","Data":"9ed3516b41f58127e294da380f02a069216b8ebfd5f998569de1b586b1314a8b"} Jul 08 17:27:50.003925 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:50.003871 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" event={"ID":"3f041301-1c23-44be-ab8e-1cd9d6f0f62a","Type":"ContainerStarted","Data":"220a91642c0994c10627078635eb57e266c9bd265deaeb172973daace7a7647e"} Jul 08 17:27:51.010840 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:51.010810 2578 generic.go:356] "Generic (PLEG): container finished" podID="d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb" containerID="d396ffe5f2ae866f0584136a06c3256ee47040dbb776e320f219026790b4f3f4" exitCode=0 Jul 08 17:27:51.011174 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:51.010878 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" event={"ID":"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb","Type":"ContainerDied","Data":"d396ffe5f2ae866f0584136a06c3256ee47040dbb776e320f219026790b4f3f4"} Jul 08 17:27:51.012470 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:51.012448 2578 generic.go:356] "Generic (PLEG): container finished" podID="3f041301-1c23-44be-ab8e-1cd9d6f0f62a" containerID="9e3d9b9af7be914d126658b4e29da606fde5c3dd0df9c099792c3e73cab6dce9" exitCode=0 Jul 08 17:27:51.012562 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:51.012542 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" event={"ID":"3f041301-1c23-44be-ab8e-1cd9d6f0f62a","Type":"ContainerDied","Data":"9e3d9b9af7be914d126658b4e29da606fde5c3dd0df9c099792c3e73cab6dce9"} Jul 08 17:27:51.014328 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:51.014308 2578 generic.go:356] "Generic (PLEG): container finished" podID="ae794944-014d-49fc-84e4-4af22f676baf" containerID="ff8817768505495ffa9e89ad0d2bf80490d7390d33f883b2003dd18ddfa0b493" exitCode=0 Jul 08 17:27:51.014422 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:51.014390 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" event={"ID":"ae794944-014d-49fc-84e4-4af22f676baf","Type":"ContainerDied","Data":"ff8817768505495ffa9e89ad0d2bf80490d7390d33f883b2003dd18ddfa0b493"} Jul 08 17:27:51.016236 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:51.016215 2578 generic.go:356] "Generic (PLEG): container finished" podID="27a036b2-a643-4d1b-a38d-9b86bd249d71" containerID="229763bb238cbd23174b53620b2ad048f98b9d475378aa85db74ec8acc645323" exitCode=0 Jul 08 17:27:51.016327 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:51.016257 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" event={"ID":"27a036b2-a643-4d1b-a38d-9b86bd249d71","Type":"ContainerDied","Data":"229763bb238cbd23174b53620b2ad048f98b9d475378aa85db74ec8acc645323"} Jul 08 17:27:52.025879 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:52.025845 2578 generic.go:356] "Generic (PLEG): container finished" podID="27a036b2-a643-4d1b-a38d-9b86bd249d71" containerID="eaf90882bf00296d67fcd3537170e04bf8c975733a700c7860155f41be109419" exitCode=0 Jul 08 17:27:52.026281 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:52.025928 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" event={"ID":"27a036b2-a643-4d1b-a38d-9b86bd249d71","Type":"ContainerDied","Data":"eaf90882bf00296d67fcd3537170e04bf8c975733a700c7860155f41be109419"} Jul 08 17:27:52.027918 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:52.027897 2578 generic.go:356] "Generic (PLEG): container finished" podID="3f041301-1c23-44be-ab8e-1cd9d6f0f62a" containerID="1eb9893f1c2bc809caba77e6ea68e1e6d2a2ee912040c71d8ced48fca5d3ff28" exitCode=0 Jul 08 17:27:52.028029 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:52.027974 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" event={"ID":"3f041301-1c23-44be-ab8e-1cd9d6f0f62a","Type":"ContainerDied","Data":"1eb9893f1c2bc809caba77e6ea68e1e6d2a2ee912040c71d8ced48fca5d3ff28"} Jul 08 17:27:52.029822 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:52.029797 2578 generic.go:356] "Generic (PLEG): container finished" podID="ae794944-014d-49fc-84e4-4af22f676baf" containerID="e9b27916f3ec74e96dff3007dc443fe9de46fc4f739412cb27b37417aaadcf26" exitCode=0 Jul 08 17:27:52.029918 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:52.029870 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" event={"ID":"ae794944-014d-49fc-84e4-4af22f676baf","Type":"ContainerDied","Data":"e9b27916f3ec74e96dff3007dc443fe9de46fc4f739412cb27b37417aaadcf26"} Jul 08 17:27:52.157844 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:52.157818 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" Jul 08 17:27:52.196222 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:52.196181 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-util\") pod \"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb\" (UID: \"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb\") " Jul 08 17:27:52.196354 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:52.196254 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-bundle\") pod \"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb\" (UID: \"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb\") " Jul 08 17:27:52.196354 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:52.196283 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9p945\" (UniqueName: \"kubernetes.io/projected/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-kube-api-access-9p945\") pod \"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb\" (UID: \"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb\") " Jul 08 17:27:52.196811 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:52.196780 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-bundle" (OuterVolumeSpecName: "bundle") pod "d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb" (UID: "d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:27:52.198405 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:52.198384 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-kube-api-access-9p945" (OuterVolumeSpecName: "kube-api-access-9p945") pod "d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb" (UID: "d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb"). InnerVolumeSpecName "kube-api-access-9p945". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:27:52.201453 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:52.201432 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-util" (OuterVolumeSpecName: "util") pod "d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb" (UID: "d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:27:52.297079 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:52.297000 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-util\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:27:52.297079 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:52.297044 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:27:52.297079 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:52.297054 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9p945\" (UniqueName: \"kubernetes.io/projected/d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb-kube-api-access-9p945\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:27:53.035792 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.035766 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" Jul 08 17:27:53.035792 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.035785 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq" event={"ID":"d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb","Type":"ContainerDied","Data":"fcf6ffd0ef8ee0062c9318feda32a14f5b0dbdd7230a4c116133c4bb9905ae1c"} Jul 08 17:27:53.036247 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.035828 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fcf6ffd0ef8ee0062c9318feda32a14f5b0dbdd7230a4c116133c4bb9905ae1c" Jul 08 17:27:53.176790 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.176768 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" Jul 08 17:27:53.202198 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.202176 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae794944-014d-49fc-84e4-4af22f676baf-bundle\") pod \"ae794944-014d-49fc-84e4-4af22f676baf\" (UID: \"ae794944-014d-49fc-84e4-4af22f676baf\") " Jul 08 17:27:53.202309 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.202249 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae794944-014d-49fc-84e4-4af22f676baf-util\") pod \"ae794944-014d-49fc-84e4-4af22f676baf\" (UID: \"ae794944-014d-49fc-84e4-4af22f676baf\") " Jul 08 17:27:53.202368 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.202335 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bm958\" (UniqueName: \"kubernetes.io/projected/ae794944-014d-49fc-84e4-4af22f676baf-kube-api-access-bm958\") pod \"ae794944-014d-49fc-84e4-4af22f676baf\" (UID: \"ae794944-014d-49fc-84e4-4af22f676baf\") " Jul 08 17:27:53.202890 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.202852 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae794944-014d-49fc-84e4-4af22f676baf-bundle" (OuterVolumeSpecName: "bundle") pod "ae794944-014d-49fc-84e4-4af22f676baf" (UID: "ae794944-014d-49fc-84e4-4af22f676baf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:27:53.204088 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.204061 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" Jul 08 17:27:53.204777 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.204753 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae794944-014d-49fc-84e4-4af22f676baf-kube-api-access-bm958" (OuterVolumeSpecName: "kube-api-access-bm958") pod "ae794944-014d-49fc-84e4-4af22f676baf" (UID: "ae794944-014d-49fc-84e4-4af22f676baf"). InnerVolumeSpecName "kube-api-access-bm958". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:27:53.206555 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.206538 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" Jul 08 17:27:53.208319 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.208294 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae794944-014d-49fc-84e4-4af22f676baf-util" (OuterVolumeSpecName: "util") pod "ae794944-014d-49fc-84e4-4af22f676baf" (UID: "ae794944-014d-49fc-84e4-4af22f676baf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:27:53.302907 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.302838 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-util\") pod \"3f041301-1c23-44be-ab8e-1cd9d6f0f62a\" (UID: \"3f041301-1c23-44be-ab8e-1cd9d6f0f62a\") " Jul 08 17:27:53.303047 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.302938 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27a036b2-a643-4d1b-a38d-9b86bd249d71-util\") pod \"27a036b2-a643-4d1b-a38d-9b86bd249d71\" (UID: \"27a036b2-a643-4d1b-a38d-9b86bd249d71\") " Jul 08 17:27:53.303047 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.302964 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-bundle\") pod \"3f041301-1c23-44be-ab8e-1cd9d6f0f62a\" (UID: \"3f041301-1c23-44be-ab8e-1cd9d6f0f62a\") " Jul 08 17:27:53.303047 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.302984 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5nt2k\" (UniqueName: \"kubernetes.io/projected/27a036b2-a643-4d1b-a38d-9b86bd249d71-kube-api-access-5nt2k\") pod \"27a036b2-a643-4d1b-a38d-9b86bd249d71\" (UID: \"27a036b2-a643-4d1b-a38d-9b86bd249d71\") " Jul 08 17:27:53.303047 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.303002 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27a036b2-a643-4d1b-a38d-9b86bd249d71-bundle\") pod \"27a036b2-a643-4d1b-a38d-9b86bd249d71\" (UID: \"27a036b2-a643-4d1b-a38d-9b86bd249d71\") " Jul 08 17:27:53.303047 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.303037 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mkln\" (UniqueName: \"kubernetes.io/projected/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-kube-api-access-9mkln\") pod \"3f041301-1c23-44be-ab8e-1cd9d6f0f62a\" (UID: \"3f041301-1c23-44be-ab8e-1cd9d6f0f62a\") " Jul 08 17:27:53.303302 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.303244 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae794944-014d-49fc-84e4-4af22f676baf-util\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:27:53.303302 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.303262 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bm958\" (UniqueName: \"kubernetes.io/projected/ae794944-014d-49fc-84e4-4af22f676baf-kube-api-access-bm958\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:27:53.303302 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.303276 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae794944-014d-49fc-84e4-4af22f676baf-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:27:53.303562 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.303521 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27a036b2-a643-4d1b-a38d-9b86bd249d71-bundle" (OuterVolumeSpecName: "bundle") pod "27a036b2-a643-4d1b-a38d-9b86bd249d71" (UID: "27a036b2-a643-4d1b-a38d-9b86bd249d71"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:27:53.303708 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.303686 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-bundle" (OuterVolumeSpecName: "bundle") pod "3f041301-1c23-44be-ab8e-1cd9d6f0f62a" (UID: "3f041301-1c23-44be-ab8e-1cd9d6f0f62a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:27:53.305072 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.305047 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-kube-api-access-9mkln" (OuterVolumeSpecName: "kube-api-access-9mkln") pod "3f041301-1c23-44be-ab8e-1cd9d6f0f62a" (UID: "3f041301-1c23-44be-ab8e-1cd9d6f0f62a"). InnerVolumeSpecName "kube-api-access-9mkln". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:27:53.305325 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.305308 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27a036b2-a643-4d1b-a38d-9b86bd249d71-kube-api-access-5nt2k" (OuterVolumeSpecName: "kube-api-access-5nt2k") pod "27a036b2-a643-4d1b-a38d-9b86bd249d71" (UID: "27a036b2-a643-4d1b-a38d-9b86bd249d71"). InnerVolumeSpecName "kube-api-access-5nt2k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:27:53.308229 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.308207 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-util" (OuterVolumeSpecName: "util") pod "3f041301-1c23-44be-ab8e-1cd9d6f0f62a" (UID: "3f041301-1c23-44be-ab8e-1cd9d6f0f62a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:27:53.308501 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.308481 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27a036b2-a643-4d1b-a38d-9b86bd249d71-util" (OuterVolumeSpecName: "util") pod "27a036b2-a643-4d1b-a38d-9b86bd249d71" (UID: "27a036b2-a643-4d1b-a38d-9b86bd249d71"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:27:53.404615 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.404578 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-util\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:27:53.404615 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.404617 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/27a036b2-a643-4d1b-a38d-9b86bd249d71-util\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:27:53.404746 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.404630 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:27:53.404746 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.404641 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5nt2k\" (UniqueName: \"kubernetes.io/projected/27a036b2-a643-4d1b-a38d-9b86bd249d71-kube-api-access-5nt2k\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:27:53.404746 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.404652 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/27a036b2-a643-4d1b-a38d-9b86bd249d71-bundle\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:27:53.404746 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:53.404660 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mkln\" (UniqueName: \"kubernetes.io/projected/3f041301-1c23-44be-ab8e-1cd9d6f0f62a-kube-api-access-9mkln\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:27:54.041838 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:54.041811 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" Jul 08 17:27:54.041838 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:54.041825 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h" event={"ID":"27a036b2-a643-4d1b-a38d-9b86bd249d71","Type":"ContainerDied","Data":"87adef69cd4b67bf57f91e750ba06e8a01164cdc302587404ac20e44e21d528a"} Jul 08 17:27:54.042351 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:54.041869 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87adef69cd4b67bf57f91e750ba06e8a01164cdc302587404ac20e44e21d528a" Jul 08 17:27:54.043666 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:54.043643 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" Jul 08 17:27:54.043782 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:54.043672 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t" event={"ID":"3f041301-1c23-44be-ab8e-1cd9d6f0f62a","Type":"ContainerDied","Data":"220a91642c0994c10627078635eb57e266c9bd265deaeb172973daace7a7647e"} Jul 08 17:27:54.043782 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:54.043712 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="220a91642c0994c10627078635eb57e266c9bd265deaeb172973daace7a7647e" Jul 08 17:27:54.045458 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:54.045440 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" event={"ID":"ae794944-014d-49fc-84e4-4af22f676baf","Type":"ContainerDied","Data":"b1347c9db0eb07d029d72be268ba3eb758ce523106f29bc6f34d07352337b91d"} Jul 08 17:27:54.045557 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:54.045468 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl" Jul 08 17:27:54.045557 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:27:54.045471 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1347c9db0eb07d029d72be268ba3eb758ce523106f29bc6f34d07352337b91d" Jul 08 17:28:19.128183 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128112 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9rx8v"] Jul 08 17:28:19.128608 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128527 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb" containerName="pull" Jul 08 17:28:19.128608 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128540 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb" containerName="pull" Jul 08 17:28:19.128608 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128553 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae794944-014d-49fc-84e4-4af22f676baf" containerName="util" Jul 08 17:28:19.128608 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128563 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae794944-014d-49fc-84e4-4af22f676baf" containerName="util" Jul 08 17:28:19.128608 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128576 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae794944-014d-49fc-84e4-4af22f676baf" containerName="pull" Jul 08 17:28:19.128608 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128599 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae794944-014d-49fc-84e4-4af22f676baf" containerName="pull" Jul 08 17:28:19.128608 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128611 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb" containerName="extract" Jul 08 17:28:19.128839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128620 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb" containerName="extract" Jul 08 17:28:19.128839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128631 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb" containerName="util" Jul 08 17:28:19.128839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128639 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb" containerName="util" Jul 08 17:28:19.128839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128648 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27a036b2-a643-4d1b-a38d-9b86bd249d71" containerName="pull" Jul 08 17:28:19.128839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128656 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="27a036b2-a643-4d1b-a38d-9b86bd249d71" containerName="pull" Jul 08 17:28:19.128839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128664 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f041301-1c23-44be-ab8e-1cd9d6f0f62a" containerName="extract" Jul 08 17:28:19.128839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128671 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f041301-1c23-44be-ab8e-1cd9d6f0f62a" containerName="extract" Jul 08 17:28:19.128839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128688 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27a036b2-a643-4d1b-a38d-9b86bd249d71" containerName="util" Jul 08 17:28:19.128839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128695 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="27a036b2-a643-4d1b-a38d-9b86bd249d71" containerName="util" Jul 08 17:28:19.128839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128704 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27a036b2-a643-4d1b-a38d-9b86bd249d71" containerName="extract" Jul 08 17:28:19.128839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128711 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="27a036b2-a643-4d1b-a38d-9b86bd249d71" containerName="extract" Jul 08 17:28:19.128839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128721 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f041301-1c23-44be-ab8e-1cd9d6f0f62a" containerName="pull" Jul 08 17:28:19.128839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128728 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f041301-1c23-44be-ab8e-1cd9d6f0f62a" containerName="pull" Jul 08 17:28:19.128839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128736 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f041301-1c23-44be-ab8e-1cd9d6f0f62a" containerName="util" Jul 08 17:28:19.128839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128743 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f041301-1c23-44be-ab8e-1cd9d6f0f62a" containerName="util" Jul 08 17:28:19.128839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128752 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae794944-014d-49fc-84e4-4af22f676baf" containerName="extract" Jul 08 17:28:19.128839 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128759 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae794944-014d-49fc-84e4-4af22f676baf" containerName="extract" Jul 08 17:28:19.129324 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128854 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f041301-1c23-44be-ab8e-1cd9d6f0f62a" containerName="extract" Jul 08 17:28:19.129324 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128864 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="27a036b2-a643-4d1b-a38d-9b86bd249d71" containerName="extract" Jul 08 17:28:19.129324 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128875 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae794944-014d-49fc-84e4-4af22f676baf" containerName="extract" Jul 08 17:28:19.129324 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.128884 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb" containerName="extract" Jul 08 17:28:19.132295 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.132278 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9rx8v" Jul 08 17:28:19.134668 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.134647 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 08 17:28:19.134868 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.134854 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 17:28:19.134922 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.134878 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-cncpd\"" Jul 08 17:28:19.135524 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.135509 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 08 17:28:19.135592 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.135537 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 17:28:19.140973 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.140952 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9rx8v"] Jul 08 17:28:19.213085 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.213056 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qclh2\" (UniqueName: \"kubernetes.io/projected/1b813e7d-1e02-4c29-b90b-d9f4f6eb910c-kube-api-access-qclh2\") pod \"kuadrant-console-plugin-66b5774985-9rx8v\" (UID: \"1b813e7d-1e02-4c29-b90b-d9f4f6eb910c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9rx8v" Jul 08 17:28:19.213310 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.213093 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1b813e7d-1e02-4c29-b90b-d9f4f6eb910c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9rx8v\" (UID: \"1b813e7d-1e02-4c29-b90b-d9f4f6eb910c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9rx8v" Jul 08 17:28:19.213310 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.213117 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b813e7d-1e02-4c29-b90b-d9f4f6eb910c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9rx8v\" (UID: \"1b813e7d-1e02-4c29-b90b-d9f4f6eb910c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9rx8v" Jul 08 17:28:19.313978 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.313945 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qclh2\" (UniqueName: \"kubernetes.io/projected/1b813e7d-1e02-4c29-b90b-d9f4f6eb910c-kube-api-access-qclh2\") pod \"kuadrant-console-plugin-66b5774985-9rx8v\" (UID: \"1b813e7d-1e02-4c29-b90b-d9f4f6eb910c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9rx8v" Jul 08 17:28:19.314133 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.313991 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1b813e7d-1e02-4c29-b90b-d9f4f6eb910c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9rx8v\" (UID: \"1b813e7d-1e02-4c29-b90b-d9f4f6eb910c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9rx8v" Jul 08 17:28:19.314133 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.314019 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b813e7d-1e02-4c29-b90b-d9f4f6eb910c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9rx8v\" (UID: \"1b813e7d-1e02-4c29-b90b-d9f4f6eb910c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9rx8v" Jul 08 17:28:19.314709 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.314677 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/1b813e7d-1e02-4c29-b90b-d9f4f6eb910c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9rx8v\" (UID: \"1b813e7d-1e02-4c29-b90b-d9f4f6eb910c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9rx8v" Jul 08 17:28:19.316392 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.316373 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b813e7d-1e02-4c29-b90b-d9f4f6eb910c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9rx8v\" (UID: \"1b813e7d-1e02-4c29-b90b-d9f4f6eb910c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9rx8v" Jul 08 17:28:19.322505 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.322479 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qclh2\" (UniqueName: \"kubernetes.io/projected/1b813e7d-1e02-4c29-b90b-d9f4f6eb910c-kube-api-access-qclh2\") pod \"kuadrant-console-plugin-66b5774985-9rx8v\" (UID: \"1b813e7d-1e02-4c29-b90b-d9f4f6eb910c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9rx8v" Jul 08 17:28:19.442175 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.442151 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9rx8v" Jul 08 17:28:19.585417 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:19.585396 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9rx8v"] Jul 08 17:28:19.587231 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:28:19.587206 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b813e7d_1e02_4c29_b90b_d9f4f6eb910c.slice/crio-ed62b336ca519f957a26e3e3676898023c92ccaac25ac5c36e4e3b9f7c2e1a10 WatchSource:0}: Error finding container ed62b336ca519f957a26e3e3676898023c92ccaac25ac5c36e4e3b9f7c2e1a10: Status 404 returned error can't find the container with id ed62b336ca519f957a26e3e3676898023c92ccaac25ac5c36e4e3b9f7c2e1a10 Jul 08 17:28:20.168799 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:20.168748 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9rx8v" event={"ID":"1b813e7d-1e02-4c29-b90b-d9f4f6eb910c","Type":"ContainerStarted","Data":"ed62b336ca519f957a26e3e3676898023c92ccaac25ac5c36e4e3b9f7c2e1a10"} Jul 08 17:28:40.284789 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:40.284713 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9rx8v" event={"ID":"1b813e7d-1e02-4c29-b90b-d9f4f6eb910c","Type":"ContainerStarted","Data":"a1d5ddaeffff072280540574ef3e2facbc352d253b1abefac4634debac7d25b8"} Jul 08 17:28:40.300642 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:28:40.300595 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9rx8v" podStartSLOduration=0.86525992 podStartE2EDuration="21.300568438s" totalImagesPullingTime="20.435308518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:28:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:28:40.299486531 +0000 UTC m=+754.525468646" watchObservedRunningTime="2026-07-08 17:28:40.300568438 +0000 UTC m=+754.526550556" Jul 08 17:29:16.648910 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:16.648872 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:29:16.651442 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:16.651419 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-52pw8" Jul 08 17:29:16.653795 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:16.653773 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 08 17:29:16.662125 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:16.662106 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:29:16.687952 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:16.687925 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:29:16.793677 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:16.793646 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9a2dfb66-3220-4b55-8a32-51deec52ac06-config-file\") pod \"limitador-limitador-54d6d988d5-52pw8\" (UID: \"9a2dfb66-3220-4b55-8a32-51deec52ac06\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-52pw8" Jul 08 17:29:16.793790 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:16.793730 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghvpv\" (UniqueName: \"kubernetes.io/projected/9a2dfb66-3220-4b55-8a32-51deec52ac06-kube-api-access-ghvpv\") pod \"limitador-limitador-54d6d988d5-52pw8\" (UID: \"9a2dfb66-3220-4b55-8a32-51deec52ac06\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-52pw8" Jul 08 17:29:16.895023 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:16.894995 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9a2dfb66-3220-4b55-8a32-51deec52ac06-config-file\") pod \"limitador-limitador-54d6d988d5-52pw8\" (UID: \"9a2dfb66-3220-4b55-8a32-51deec52ac06\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-52pw8" Jul 08 17:29:16.895168 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:16.895149 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghvpv\" (UniqueName: \"kubernetes.io/projected/9a2dfb66-3220-4b55-8a32-51deec52ac06-kube-api-access-ghvpv\") pod \"limitador-limitador-54d6d988d5-52pw8\" (UID: \"9a2dfb66-3220-4b55-8a32-51deec52ac06\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-52pw8" Jul 08 17:29:16.895634 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:16.895616 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9a2dfb66-3220-4b55-8a32-51deec52ac06-config-file\") pod \"limitador-limitador-54d6d988d5-52pw8\" (UID: \"9a2dfb66-3220-4b55-8a32-51deec52ac06\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-52pw8" Jul 08 17:29:16.902826 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:16.902769 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghvpv\" (UniqueName: \"kubernetes.io/projected/9a2dfb66-3220-4b55-8a32-51deec52ac06-kube-api-access-ghvpv\") pod \"limitador-limitador-54d6d988d5-52pw8\" (UID: \"9a2dfb66-3220-4b55-8a32-51deec52ac06\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-52pw8" Jul 08 17:29:16.961396 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:16.961362 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-52pw8" Jul 08 17:29:17.091471 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:17.091443 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:29:17.093157 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:29:17.093129 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a2dfb66_3220_4b55_8a32_51deec52ac06.slice/crio-a7bc5e106189ddb4245b38dc727c6548066b04f78f157557dd6d052f81f8bf2a WatchSource:0}: Error finding container a7bc5e106189ddb4245b38dc727c6548066b04f78f157557dd6d052f81f8bf2a: Status 404 returned error can't find the container with id a7bc5e106189ddb4245b38dc727c6548066b04f78f157557dd6d052f81f8bf2a Jul 08 17:29:17.356042 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:17.356015 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-4wk92"] Jul 08 17:29:17.358917 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:17.358901 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4wk92" Jul 08 17:29:17.361338 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:17.361321 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-x5klb\"" Jul 08 17:29:17.366466 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:17.366441 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4wk92"] Jul 08 17:29:17.462383 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:17.462353 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-52pw8" event={"ID":"9a2dfb66-3220-4b55-8a32-51deec52ac06","Type":"ContainerStarted","Data":"a7bc5e106189ddb4245b38dc727c6548066b04f78f157557dd6d052f81f8bf2a"} Jul 08 17:29:17.499814 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:17.499784 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qv6x\" (UniqueName: \"kubernetes.io/projected/37f51bb0-03d8-4370-81fc-8ef13c0c4e3d-kube-api-access-2qv6x\") pod \"authorino-75d556bd77-4wk92\" (UID: \"37f51bb0-03d8-4370-81fc-8ef13c0c4e3d\") " pod="kuadrant-system/authorino-75d556bd77-4wk92" Jul 08 17:29:17.601133 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:17.601103 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qv6x\" (UniqueName: \"kubernetes.io/projected/37f51bb0-03d8-4370-81fc-8ef13c0c4e3d-kube-api-access-2qv6x\") pod \"authorino-75d556bd77-4wk92\" (UID: \"37f51bb0-03d8-4370-81fc-8ef13c0c4e3d\") " pod="kuadrant-system/authorino-75d556bd77-4wk92" Jul 08 17:29:17.609214 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:17.609160 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qv6x\" (UniqueName: \"kubernetes.io/projected/37f51bb0-03d8-4370-81fc-8ef13c0c4e3d-kube-api-access-2qv6x\") pod \"authorino-75d556bd77-4wk92\" (UID: \"37f51bb0-03d8-4370-81fc-8ef13c0c4e3d\") " pod="kuadrant-system/authorino-75d556bd77-4wk92" Jul 08 17:29:17.670367 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:17.670346 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4wk92" Jul 08 17:29:17.844713 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:17.844646 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4wk92"] Jul 08 17:29:17.858036 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:29:17.858008 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37f51bb0_03d8_4370_81fc_8ef13c0c4e3d.slice/crio-e355e31cf819786786096fcd9735dced51fac79bcede03d09bb898c2eea37e78 WatchSource:0}: Error finding container e355e31cf819786786096fcd9735dced51fac79bcede03d09bb898c2eea37e78: Status 404 returned error can't find the container with id e355e31cf819786786096fcd9735dced51fac79bcede03d09bb898c2eea37e78 Jul 08 17:29:18.469386 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:18.469345 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4wk92" event={"ID":"37f51bb0-03d8-4370-81fc-8ef13c0c4e3d","Type":"ContainerStarted","Data":"e355e31cf819786786096fcd9735dced51fac79bcede03d09bb898c2eea37e78"} Jul 08 17:29:20.489654 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:20.489612 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-52pw8" event={"ID":"9a2dfb66-3220-4b55-8a32-51deec52ac06","Type":"ContainerStarted","Data":"409bc6e0c3ad81613f6d9b31fcb6e0464fdc3a9f6553b1ae999f3a23f50bf3b7"} Jul 08 17:29:20.490069 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:20.489714 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-52pw8" Jul 08 17:29:20.507256 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:20.507210 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-52pw8" podStartSLOduration=1.84962435 podStartE2EDuration="4.507190101s" totalImagesPullingTime="2.657565751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:29:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:29:20.50506747 +0000 UTC m=+794.731049612" watchObservedRunningTime="2026-07-08 17:29:20.507190101 +0000 UTC m=+794.733172223" Jul 08 17:29:22.501159 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:22.501121 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4wk92" event={"ID":"37f51bb0-03d8-4370-81fc-8ef13c0c4e3d","Type":"ContainerStarted","Data":"a06062dadac563cdcaa8e45a813809afa975a45bd3a82c17cba4d229f8e5bae6"} Jul 08 17:29:22.517702 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:22.517647 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-4wk92" podStartSLOduration=1.463010842 podStartE2EDuration="5.517630599s" totalImagesPullingTime="4.054619757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:29:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:29:22.51591475 +0000 UTC m=+796.741896874" watchObservedRunningTime="2026-07-08 17:29:22.517630599 +0000 UTC m=+796.743612719" Jul 08 17:29:31.495788 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:31.495710 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-52pw8" Jul 08 17:29:49.189606 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.189551 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m6kd8"] Jul 08 17:29:49.259658 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.259630 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m6kd8"] Jul 08 17:29:49.259818 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.259720 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-m6kd8" Jul 08 17:29:49.370838 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.370804 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdjlw\" (UniqueName: \"kubernetes.io/projected/187d0d40-7648-464f-8051-d0416d5d925e-kube-api-access-qdjlw\") pod \"authorino-5d4b67c446-m6kd8\" (UID: \"187d0d40-7648-464f-8051-d0416d5d925e\") " pod="kuadrant-system/authorino-5d4b67c446-m6kd8" Jul 08 17:29:49.447267 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.447187 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m6kd8"] Jul 08 17:29:49.447462 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:29:49.447442 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-qdjlw], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-m6kd8" podUID="187d0d40-7648-464f-8051-d0416d5d925e" Jul 08 17:29:49.471374 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.471336 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdjlw\" (UniqueName: \"kubernetes.io/projected/187d0d40-7648-464f-8051-d0416d5d925e-kube-api-access-qdjlw\") pod \"authorino-5d4b67c446-m6kd8\" (UID: \"187d0d40-7648-464f-8051-d0416d5d925e\") " pod="kuadrant-system/authorino-5d4b67c446-m6kd8" Jul 08 17:29:49.472371 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.472350 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-96bf87968-tx9rs"] Jul 08 17:29:49.479530 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.479507 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdjlw\" (UniqueName: \"kubernetes.io/projected/187d0d40-7648-464f-8051-d0416d5d925e-kube-api-access-qdjlw\") pod \"authorino-5d4b67c446-m6kd8\" (UID: \"187d0d40-7648-464f-8051-d0416d5d925e\") " pod="kuadrant-system/authorino-5d4b67c446-m6kd8" Jul 08 17:29:49.491991 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.491969 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-96bf87968-tx9rs"] Jul 08 17:29:49.492103 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.492055 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-96bf87968-tx9rs" Jul 08 17:29:49.494550 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.494532 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 08 17:29:49.572283 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.572247 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkc4z\" (UniqueName: \"kubernetes.io/projected/78ba7499-90ea-487d-a3b3-7785891ac2e4-kube-api-access-rkc4z\") pod \"authorino-96bf87968-tx9rs\" (UID: \"78ba7499-90ea-487d-a3b3-7785891ac2e4\") " pod="kuadrant-system/authorino-96bf87968-tx9rs" Jul 08 17:29:49.572435 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.572318 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78ba7499-90ea-487d-a3b3-7785891ac2e4-tls-cert\") pod \"authorino-96bf87968-tx9rs\" (UID: \"78ba7499-90ea-487d-a3b3-7785891ac2e4\") " pod="kuadrant-system/authorino-96bf87968-tx9rs" Jul 08 17:29:49.634115 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.634087 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-m6kd8" Jul 08 17:29:49.652440 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.652414 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-m6kd8" Jul 08 17:29:49.673687 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.673662 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdjlw\" (UniqueName: \"kubernetes.io/projected/187d0d40-7648-464f-8051-d0416d5d925e-kube-api-access-qdjlw\") pod \"187d0d40-7648-464f-8051-d0416d5d925e\" (UID: \"187d0d40-7648-464f-8051-d0416d5d925e\") " Jul 08 17:29:49.673800 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.673768 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78ba7499-90ea-487d-a3b3-7785891ac2e4-tls-cert\") pod \"authorino-96bf87968-tx9rs\" (UID: \"78ba7499-90ea-487d-a3b3-7785891ac2e4\") " pod="kuadrant-system/authorino-96bf87968-tx9rs" Jul 08 17:29:49.673838 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.673800 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkc4z\" (UniqueName: \"kubernetes.io/projected/78ba7499-90ea-487d-a3b3-7785891ac2e4-kube-api-access-rkc4z\") pod \"authorino-96bf87968-tx9rs\" (UID: \"78ba7499-90ea-487d-a3b3-7785891ac2e4\") " pod="kuadrant-system/authorino-96bf87968-tx9rs" Jul 08 17:29:49.675840 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.675809 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/187d0d40-7648-464f-8051-d0416d5d925e-kube-api-access-qdjlw" (OuterVolumeSpecName: "kube-api-access-qdjlw") pod "187d0d40-7648-464f-8051-d0416d5d925e" (UID: "187d0d40-7648-464f-8051-d0416d5d925e"). InnerVolumeSpecName "kube-api-access-qdjlw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:29:49.676064 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.676047 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78ba7499-90ea-487d-a3b3-7785891ac2e4-tls-cert\") pod \"authorino-96bf87968-tx9rs\" (UID: \"78ba7499-90ea-487d-a3b3-7785891ac2e4\") " pod="kuadrant-system/authorino-96bf87968-tx9rs" Jul 08 17:29:49.681234 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.681212 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkc4z\" (UniqueName: \"kubernetes.io/projected/78ba7499-90ea-487d-a3b3-7785891ac2e4-kube-api-access-rkc4z\") pod \"authorino-96bf87968-tx9rs\" (UID: \"78ba7499-90ea-487d-a3b3-7785891ac2e4\") " pod="kuadrant-system/authorino-96bf87968-tx9rs" Jul 08 17:29:49.774504 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.774427 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdjlw\" (UniqueName: \"kubernetes.io/projected/187d0d40-7648-464f-8051-d0416d5d925e-kube-api-access-qdjlw\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:29:49.801222 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.801186 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-96bf87968-tx9rs" Jul 08 17:29:49.948610 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:49.948557 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-96bf87968-tx9rs"] Jul 08 17:29:49.952041 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:29:49.952015 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78ba7499_90ea_487d_a3b3_7785891ac2e4.slice/crio-c2d8f8fbf70c76848e2fee3a80a0ead5c94a59b08a1bf14eacb17722177cf0e5 WatchSource:0}: Error finding container c2d8f8fbf70c76848e2fee3a80a0ead5c94a59b08a1bf14eacb17722177cf0e5: Status 404 returned error can't find the container with id c2d8f8fbf70c76848e2fee3a80a0ead5c94a59b08a1bf14eacb17722177cf0e5 Jul 08 17:29:50.641843 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:50.641803 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-96bf87968-tx9rs" event={"ID":"78ba7499-90ea-487d-a3b3-7785891ac2e4","Type":"ContainerStarted","Data":"15c1b3f10da2e10a66883d1ad04d363cd87606714e71a000eb138e49537db010"} Jul 08 17:29:50.642233 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:50.641858 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-96bf87968-tx9rs" event={"ID":"78ba7499-90ea-487d-a3b3-7785891ac2e4","Type":"ContainerStarted","Data":"c2d8f8fbf70c76848e2fee3a80a0ead5c94a59b08a1bf14eacb17722177cf0e5"} Jul 08 17:29:50.642233 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:50.641892 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-m6kd8" Jul 08 17:29:50.657841 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:50.657736 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-96bf87968-tx9rs" podStartSLOduration=1.083838008 podStartE2EDuration="1.657720387s" totalImagesPullingTime="573.882379ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:29:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:29:50.656414626 +0000 UTC m=+824.882396747" watchObservedRunningTime="2026-07-08 17:29:50.657720387 +0000 UTC m=+824.883702507" Jul 08 17:29:50.682727 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:50.682697 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4wk92"] Jul 08 17:29:50.682913 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:50.682877 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-4wk92" podUID="37f51bb0-03d8-4370-81fc-8ef13c0c4e3d" containerName="authorino" containerID="cri-o://a06062dadac563cdcaa8e45a813809afa975a45bd3a82c17cba4d229f8e5bae6" gracePeriod=30 Jul 08 17:29:50.685884 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:50.685618 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m6kd8"] Jul 08 17:29:50.692577 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:50.692523 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-m6kd8"] Jul 08 17:29:50.908131 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:50.908076 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4wk92" Jul 08 17:29:50.983255 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:50.983221 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2qv6x\" (UniqueName: \"kubernetes.io/projected/37f51bb0-03d8-4370-81fc-8ef13c0c4e3d-kube-api-access-2qv6x\") pod \"37f51bb0-03d8-4370-81fc-8ef13c0c4e3d\" (UID: \"37f51bb0-03d8-4370-81fc-8ef13c0c4e3d\") " Jul 08 17:29:50.985197 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:50.985166 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37f51bb0-03d8-4370-81fc-8ef13c0c4e3d-kube-api-access-2qv6x" (OuterVolumeSpecName: "kube-api-access-2qv6x") pod "37f51bb0-03d8-4370-81fc-8ef13c0c4e3d" (UID: "37f51bb0-03d8-4370-81fc-8ef13c0c4e3d"). InnerVolumeSpecName "kube-api-access-2qv6x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:29:51.084666 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:51.084630 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2qv6x\" (UniqueName: \"kubernetes.io/projected/37f51bb0-03d8-4370-81fc-8ef13c0c4e3d-kube-api-access-2qv6x\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:29:51.647725 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:51.647689 2578 generic.go:356] "Generic (PLEG): container finished" podID="37f51bb0-03d8-4370-81fc-8ef13c0c4e3d" containerID="a06062dadac563cdcaa8e45a813809afa975a45bd3a82c17cba4d229f8e5bae6" exitCode=0 Jul 08 17:29:51.648159 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:51.647734 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4wk92" Jul 08 17:29:51.648159 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:51.647773 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4wk92" event={"ID":"37f51bb0-03d8-4370-81fc-8ef13c0c4e3d","Type":"ContainerDied","Data":"a06062dadac563cdcaa8e45a813809afa975a45bd3a82c17cba4d229f8e5bae6"} Jul 08 17:29:51.648159 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:51.647825 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4wk92" event={"ID":"37f51bb0-03d8-4370-81fc-8ef13c0c4e3d","Type":"ContainerDied","Data":"e355e31cf819786786096fcd9735dced51fac79bcede03d09bb898c2eea37e78"} Jul 08 17:29:51.648159 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:51.647855 2578 scope.go:117] "RemoveContainer" containerID="a06062dadac563cdcaa8e45a813809afa975a45bd3a82c17cba4d229f8e5bae6" Jul 08 17:29:51.656668 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:51.656652 2578 scope.go:117] "RemoveContainer" containerID="a06062dadac563cdcaa8e45a813809afa975a45bd3a82c17cba4d229f8e5bae6" Jul 08 17:29:51.656914 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:29:51.656898 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a06062dadac563cdcaa8e45a813809afa975a45bd3a82c17cba4d229f8e5bae6\": container with ID starting with a06062dadac563cdcaa8e45a813809afa975a45bd3a82c17cba4d229f8e5bae6 not found: ID does not exist" containerID="a06062dadac563cdcaa8e45a813809afa975a45bd3a82c17cba4d229f8e5bae6" Jul 08 17:29:51.656959 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:51.656922 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a06062dadac563cdcaa8e45a813809afa975a45bd3a82c17cba4d229f8e5bae6"} err="failed to get container status \"a06062dadac563cdcaa8e45a813809afa975a45bd3a82c17cba4d229f8e5bae6\": rpc error: code = NotFound desc = could not find container \"a06062dadac563cdcaa8e45a813809afa975a45bd3a82c17cba4d229f8e5bae6\": container with ID starting with a06062dadac563cdcaa8e45a813809afa975a45bd3a82c17cba4d229f8e5bae6 not found: ID does not exist" Jul 08 17:29:51.668296 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:51.668271 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4wk92"] Jul 08 17:29:51.671869 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:51.671849 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4wk92"] Jul 08 17:29:52.393910 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:52.393880 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="187d0d40-7648-464f-8051-d0416d5d925e" path="/var/lib/kubelet/pods/187d0d40-7648-464f-8051-d0416d5d925e/volumes" Jul 08 17:29:52.394111 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:29:52.394099 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37f51bb0-03d8-4370-81fc-8ef13c0c4e3d" path="/var/lib/kubelet/pods/37f51bb0-03d8-4370-81fc-8ef13c0c4e3d/volumes" Jul 08 17:30:31.483200 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:30:31.483161 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:30:53.640507 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:30:53.640467 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:31:01.642119 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:31:01.642033 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:31:06.849822 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:31:06.849775 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:31:15.644176 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:31:15.644142 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:32:05.140323 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:05.140287 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:32:12.660659 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.659682 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7"] Jul 08 17:32:12.660659 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.660399 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37f51bb0-03d8-4370-81fc-8ef13c0c4e3d" containerName="authorino" Jul 08 17:32:12.660659 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.660419 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="37f51bb0-03d8-4370-81fc-8ef13c0c4e3d" containerName="authorino" Jul 08 17:32:12.662742 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.661252 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="37f51bb0-03d8-4370-81fc-8ef13c0c4e3d" containerName="authorino" Jul 08 17:32:12.664861 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.664835 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.668553 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.668525 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 17:32:12.668698 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.668605 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 08 17:32:12.668698 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.668531 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mzzmx\"" Jul 08 17:32:12.668698 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.668630 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 17:32:12.674971 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.674948 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7"] Jul 08 17:32:12.750842 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.750801 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/184f0973-0787-4ff2-bcc0-cb08c24a6fda-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.750842 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.750841 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rs2l\" (UniqueName: \"kubernetes.io/projected/184f0973-0787-4ff2-bcc0-cb08c24a6fda-kube-api-access-7rs2l\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.751054 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.750871 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/184f0973-0787-4ff2-bcc0-cb08c24a6fda-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.751054 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.750995 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/184f0973-0787-4ff2-bcc0-cb08c24a6fda-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.751054 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.751036 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/184f0973-0787-4ff2-bcc0-cb08c24a6fda-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.751153 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.751088 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/184f0973-0787-4ff2-bcc0-cb08c24a6fda-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.852347 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.852315 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/184f0973-0787-4ff2-bcc0-cb08c24a6fda-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.852515 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.852356 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/184f0973-0787-4ff2-bcc0-cb08c24a6fda-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.852515 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.852380 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/184f0973-0787-4ff2-bcc0-cb08c24a6fda-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.852515 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.852415 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/184f0973-0787-4ff2-bcc0-cb08c24a6fda-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.852515 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.852432 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rs2l\" (UniqueName: \"kubernetes.io/projected/184f0973-0787-4ff2-bcc0-cb08c24a6fda-kube-api-access-7rs2l\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.852515 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.852451 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/184f0973-0787-4ff2-bcc0-cb08c24a6fda-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.852818 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.852795 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/184f0973-0787-4ff2-bcc0-cb08c24a6fda-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.852897 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.852873 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/184f0973-0787-4ff2-bcc0-cb08c24a6fda-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.852950 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.852924 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/184f0973-0787-4ff2-bcc0-cb08c24a6fda-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.854669 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.854651 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/184f0973-0787-4ff2-bcc0-cb08c24a6fda-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.854930 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.854913 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/184f0973-0787-4ff2-bcc0-cb08c24a6fda-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.860693 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.860671 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rs2l\" (UniqueName: \"kubernetes.io/projected/184f0973-0787-4ff2-bcc0-cb08c24a6fda-kube-api-access-7rs2l\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7\" (UID: \"184f0973-0787-4ff2-bcc0-cb08c24a6fda\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:12.975031 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:12.974996 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:13.104446 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:13.104422 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7"] Jul 08 17:32:13.106323 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:32:13.106295 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod184f0973_0787_4ff2_bcc0_cb08c24a6fda.slice/crio-61b3e0b56b861bdb1f88cb9d3674e7d5ca0c23e871614aa83056f9c29ee083c9 WatchSource:0}: Error finding container 61b3e0b56b861bdb1f88cb9d3674e7d5ca0c23e871614aa83056f9c29ee083c9: Status 404 returned error can't find the container with id 61b3e0b56b861bdb1f88cb9d3674e7d5ca0c23e871614aa83056f9c29ee083c9 Jul 08 17:32:13.107690 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:13.107673 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 17:32:13.360766 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:13.360662 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" event={"ID":"184f0973-0787-4ff2-bcc0-cb08c24a6fda","Type":"ContainerStarted","Data":"61b3e0b56b861bdb1f88cb9d3674e7d5ca0c23e871614aa83056f9c29ee083c9"} Jul 08 17:32:14.366820 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:14.366784 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" event={"ID":"184f0973-0787-4ff2-bcc0-cb08c24a6fda","Type":"ContainerStarted","Data":"4806d6334a3471e545907e0a4d3e6cdf71b2af49063e9e5811bbd79b064e1a89"} Jul 08 17:32:14.385957 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:14.385871 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" podStartSLOduration=1.358220226 podStartE2EDuration="2.385857069s" totalImagesPullingTime="1.027636843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:32:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:32:14.384025099 +0000 UTC m=+968.610007238" watchObservedRunningTime="2026-07-08 17:32:14.385857069 +0000 UTC m=+968.611839188" Jul 08 17:32:14.956579 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:14.956544 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:32:22.975824 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:22.975789 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:22.976278 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:22.975845 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:22.988504 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:22.988482 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:32:23.424124 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:32:23.424043 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7" Jul 08 17:33:01.845568 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:01.845535 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57766c5757-zt6pp"] Jul 08 17:33:01.849009 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:01.848991 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57766c5757-zt6pp" Jul 08 17:33:01.858910 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:01.858886 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57766c5757-zt6pp"] Jul 08 17:33:01.954990 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:01.954956 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad2ee5ab-dceb-4b51-9547-5a05f443e29c-tls-cert\") pod \"authorino-57766c5757-zt6pp\" (UID: \"ad2ee5ab-dceb-4b51-9547-5a05f443e29c\") " pod="kuadrant-system/authorino-57766c5757-zt6pp" Jul 08 17:33:01.955154 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:01.955010 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncgdq\" (UniqueName: \"kubernetes.io/projected/ad2ee5ab-dceb-4b51-9547-5a05f443e29c-kube-api-access-ncgdq\") pod \"authorino-57766c5757-zt6pp\" (UID: \"ad2ee5ab-dceb-4b51-9547-5a05f443e29c\") " pod="kuadrant-system/authorino-57766c5757-zt6pp" Jul 08 17:33:02.055817 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:02.055778 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncgdq\" (UniqueName: \"kubernetes.io/projected/ad2ee5ab-dceb-4b51-9547-5a05f443e29c-kube-api-access-ncgdq\") pod \"authorino-57766c5757-zt6pp\" (UID: \"ad2ee5ab-dceb-4b51-9547-5a05f443e29c\") " pod="kuadrant-system/authorino-57766c5757-zt6pp" Jul 08 17:33:02.055936 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:02.055871 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad2ee5ab-dceb-4b51-9547-5a05f443e29c-tls-cert\") pod \"authorino-57766c5757-zt6pp\" (UID: \"ad2ee5ab-dceb-4b51-9547-5a05f443e29c\") " pod="kuadrant-system/authorino-57766c5757-zt6pp" Jul 08 17:33:02.058344 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:02.058324 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ad2ee5ab-dceb-4b51-9547-5a05f443e29c-tls-cert\") pod \"authorino-57766c5757-zt6pp\" (UID: \"ad2ee5ab-dceb-4b51-9547-5a05f443e29c\") " pod="kuadrant-system/authorino-57766c5757-zt6pp" Jul 08 17:33:02.064247 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:02.064223 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncgdq\" (UniqueName: \"kubernetes.io/projected/ad2ee5ab-dceb-4b51-9547-5a05f443e29c-kube-api-access-ncgdq\") pod \"authorino-57766c5757-zt6pp\" (UID: \"ad2ee5ab-dceb-4b51-9547-5a05f443e29c\") " pod="kuadrant-system/authorino-57766c5757-zt6pp" Jul 08 17:33:02.157895 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:02.157807 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57766c5757-zt6pp" Jul 08 17:33:02.305264 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:02.305234 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57766c5757-zt6pp"] Jul 08 17:33:02.306644 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:33:02.306579 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad2ee5ab_dceb_4b51_9547_5a05f443e29c.slice/crio-0b41370afac2b7cd89a26ff448d05b5e1e9334ac85b818d57498eb146e9444fa WatchSource:0}: Error finding container 0b41370afac2b7cd89a26ff448d05b5e1e9334ac85b818d57498eb146e9444fa: Status 404 returned error can't find the container with id 0b41370afac2b7cd89a26ff448d05b5e1e9334ac85b818d57498eb146e9444fa Jul 08 17:33:02.599395 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:02.599361 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57766c5757-zt6pp" event={"ID":"ad2ee5ab-dceb-4b51-9547-5a05f443e29c","Type":"ContainerStarted","Data":"0b41370afac2b7cd89a26ff448d05b5e1e9334ac85b818d57498eb146e9444fa"} Jul 08 17:33:03.608271 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:03.608217 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57766c5757-zt6pp" event={"ID":"ad2ee5ab-dceb-4b51-9547-5a05f443e29c","Type":"ContainerStarted","Data":"4a266b048c0e575cd92c071978fbd3d1170dcd8b688d29f0179a8b989b1dd95e"} Jul 08 17:33:03.632515 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:03.632451 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-57766c5757-zt6pp" podStartSLOduration=2.296258753 podStartE2EDuration="2.632431842s" totalImagesPullingTime="336.173089ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:33:03.630716359 +0000 UTC m=+1017.856698476" watchObservedRunningTime="2026-07-08 17:33:03.632431842 +0000 UTC m=+1017.858413963" Jul 08 17:33:03.673761 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:03.673727 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-96bf87968-tx9rs"] Jul 08 17:33:03.674000 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:03.673967 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-96bf87968-tx9rs" podUID="78ba7499-90ea-487d-a3b3-7785891ac2e4" containerName="authorino" containerID="cri-o://15c1b3f10da2e10a66883d1ad04d363cd87606714e71a000eb138e49537db010" gracePeriod=30 Jul 08 17:33:03.935378 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:03.935354 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-96bf87968-tx9rs" Jul 08 17:33:03.975546 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:03.975506 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkc4z\" (UniqueName: \"kubernetes.io/projected/78ba7499-90ea-487d-a3b3-7785891ac2e4-kube-api-access-rkc4z\") pod \"78ba7499-90ea-487d-a3b3-7785891ac2e4\" (UID: \"78ba7499-90ea-487d-a3b3-7785891ac2e4\") " Jul 08 17:33:03.975754 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:03.975642 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78ba7499-90ea-487d-a3b3-7785891ac2e4-tls-cert\") pod \"78ba7499-90ea-487d-a3b3-7785891ac2e4\" (UID: \"78ba7499-90ea-487d-a3b3-7785891ac2e4\") " Jul 08 17:33:03.977809 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:03.977779 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78ba7499-90ea-487d-a3b3-7785891ac2e4-kube-api-access-rkc4z" (OuterVolumeSpecName: "kube-api-access-rkc4z") pod "78ba7499-90ea-487d-a3b3-7785891ac2e4" (UID: "78ba7499-90ea-487d-a3b3-7785891ac2e4"). InnerVolumeSpecName "kube-api-access-rkc4z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:33:03.988293 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:03.988250 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78ba7499-90ea-487d-a3b3-7785891ac2e4-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "78ba7499-90ea-487d-a3b3-7785891ac2e4" (UID: "78ba7499-90ea-487d-a3b3-7785891ac2e4"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:33:04.076647 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:04.076614 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkc4z\" (UniqueName: \"kubernetes.io/projected/78ba7499-90ea-487d-a3b3-7785891ac2e4-kube-api-access-rkc4z\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:33:04.076647 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:04.076642 2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78ba7499-90ea-487d-a3b3-7785891ac2e4-tls-cert\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:33:04.615877 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:04.615835 2578 generic.go:356] "Generic (PLEG): container finished" podID="78ba7499-90ea-487d-a3b3-7785891ac2e4" containerID="15c1b3f10da2e10a66883d1ad04d363cd87606714e71a000eb138e49537db010" exitCode=0 Jul 08 17:33:04.616311 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:04.615945 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-96bf87968-tx9rs" Jul 08 17:33:04.616311 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:04.615982 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-96bf87968-tx9rs" event={"ID":"78ba7499-90ea-487d-a3b3-7785891ac2e4","Type":"ContainerDied","Data":"15c1b3f10da2e10a66883d1ad04d363cd87606714e71a000eb138e49537db010"} Jul 08 17:33:04.616311 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:04.616032 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-96bf87968-tx9rs" event={"ID":"78ba7499-90ea-487d-a3b3-7785891ac2e4","Type":"ContainerDied","Data":"c2d8f8fbf70c76848e2fee3a80a0ead5c94a59b08a1bf14eacb17722177cf0e5"} Jul 08 17:33:04.616311 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:04.616058 2578 scope.go:117] "RemoveContainer" containerID="15c1b3f10da2e10a66883d1ad04d363cd87606714e71a000eb138e49537db010" Jul 08 17:33:04.629499 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:04.629480 2578 scope.go:117] "RemoveContainer" containerID="15c1b3f10da2e10a66883d1ad04d363cd87606714e71a000eb138e49537db010" Jul 08 17:33:04.629793 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:33:04.629773 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15c1b3f10da2e10a66883d1ad04d363cd87606714e71a000eb138e49537db010\": container with ID starting with 15c1b3f10da2e10a66883d1ad04d363cd87606714e71a000eb138e49537db010 not found: ID does not exist" containerID="15c1b3f10da2e10a66883d1ad04d363cd87606714e71a000eb138e49537db010" Jul 08 17:33:04.629865 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:04.629839 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15c1b3f10da2e10a66883d1ad04d363cd87606714e71a000eb138e49537db010"} err="failed to get container status \"15c1b3f10da2e10a66883d1ad04d363cd87606714e71a000eb138e49537db010\": rpc error: code = NotFound desc = could not find container \"15c1b3f10da2e10a66883d1ad04d363cd87606714e71a000eb138e49537db010\": container with ID starting with 15c1b3f10da2e10a66883d1ad04d363cd87606714e71a000eb138e49537db010 not found: ID does not exist" Jul 08 17:33:04.642955 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:04.642928 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-96bf87968-tx9rs"] Jul 08 17:33:04.649315 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:04.649294 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-96bf87968-tx9rs"] Jul 08 17:33:06.394532 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:06.394489 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78ba7499-90ea-487d-a3b3-7785891ac2e4" path="/var/lib/kubelet/pods/78ba7499-90ea-487d-a3b3-7785891ac2e4/volumes" Jul 08 17:33:33.346275 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:33.346240 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:33:42.941715 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:42.941679 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:33:55.440706 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:33:55.440650 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:34:03.642900 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:34:03.642815 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:34:14.641892 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:34:14.641862 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:34:42.545738 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:34:42.545705 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:34:55.035575 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:34:55.035539 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:35:31.643085 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:35:31.643013 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:35:40.943168 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:35:40.943131 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:35:49.943274 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:35:49.943236 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:36:01.143790 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:36:01.143750 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:36:10.638547 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:36:10.638511 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:36:22.255753 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:36:22.255720 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:36:33.341744 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:36:33.341704 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:36:43.143773 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:36:43.143736 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:37:06.246000 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:37:06.245919 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:37:18.144884 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:37:18.144839 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:37:41.498783 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:37:41.498754 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:37:52.136225 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:37:52.136193 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:38:07.647224 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:38:07.647179 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:38:17.238326 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:38:17.238280 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:38:28.235685 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:38:28.235650 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:38:38.040031 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:38:38.039991 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:38:47.740963 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:38:47.740930 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:40:34.252927 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:40:34.252888 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:40:42.449992 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:40:42.449954 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:40:55.343321 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:40:55.343285 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:41:05.742631 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:41:05.742579 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:41:18.147740 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:41:18.147708 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:41:27.645947 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:41:27.645910 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:41:36.741145 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:41:36.741113 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:41:48.744076 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:41:48.744043 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:41:58.146201 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:41:58.146167 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:42:08.942639 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:42:08.942597 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:42:20.841498 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:42:20.841466 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:42:43.341648 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:42:43.341612 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:42:52.754039 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:42:52.754006 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:43:02.748406 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:43:02.748316 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:43:12.442230 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:43:12.442198 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:43:19.440825 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:43:19.440792 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:43:35.248664 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:43:35.248622 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:43:45.354318 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:43:45.354285 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:43:54.732961 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:43:54.732927 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:44:03.240897 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:44:03.240860 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:44:14.450631 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:44:14.450596 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:44:26.146877 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:44:26.146844 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:44:42.748162 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:44:42.748132 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:44:53.839797 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:44:53.839759 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:45:08.642425 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:45:08.642390 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:45:21.045352 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:45:21.045322 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:45:36.539258 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:45:36.539228 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:45:49.043768 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:45:49.043725 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:45:58.938096 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:45:58.938019 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:46:13.136114 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:46:13.136078 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:46:22.351190 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:46:22.351159 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:46:33.848400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:46:33.848364 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:46:48.992080 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:46:48.992043 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:47:01.346433 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:47:01.346403 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:47:08.153064 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:47:08.153031 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:47:19.143982 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:47:19.143948 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:48:39.648079 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:48:39.648040 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:48:49.138415 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:48:49.138376 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:49:00.939566 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:00.939480 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:49:11.448892 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:11.448849 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:49:40.752654 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:40.752617 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:49:57.339004 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.338964 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d"] Jul 08 17:49:57.340050 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.339388 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78ba7499-90ea-487d-a3b3-7785891ac2e4" containerName="authorino" Jul 08 17:49:57.340050 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.339408 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="78ba7499-90ea-487d-a3b3-7785891ac2e4" containerName="authorino" Jul 08 17:49:57.340050 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.339487 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="78ba7499-90ea-487d-a3b3-7785891ac2e4" containerName="authorino" Jul 08 17:49:57.342578 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.342558 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.345745 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.345719 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-525288bb\"/\"openshift-service-ca.crt\"" Jul 08 17:49:57.345871 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.345779 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-525288bb\"/\"default-dockercfg-sgnbd\"" Jul 08 17:49:57.345871 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.345725 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-525288bb\"/\"e2e-lifecycle-model-525288bb-kserve-self-signed-certs\"" Jul 08 17:49:57.346019 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.345994 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-525288bb\"/\"kube-root-ca.crt\"" Jul 08 17:49:57.353901 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.353875 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d"] Jul 08 17:49:57.382030 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.381996 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw9tq\" (UniqueName: \"kubernetes.io/projected/98616b96-63db-43fa-82bf-7839bbf063b5-kube-api-access-hw9tq\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.382030 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.382033 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-tmp-dir\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.382254 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.382063 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-dshm\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.382254 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.382085 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98616b96-63db-43fa-82bf-7839bbf063b5-tls-certs\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.382254 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.382149 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-home\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.382254 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.382185 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-model-cache\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.482596 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.482542 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-home\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.482784 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.482629 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-model-cache\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.482784 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.482705 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw9tq\" (UniqueName: \"kubernetes.io/projected/98616b96-63db-43fa-82bf-7839bbf063b5-kube-api-access-hw9tq\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.482784 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.482748 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-tmp-dir\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.482956 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.482787 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-dshm\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.482956 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.482808 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98616b96-63db-43fa-82bf-7839bbf063b5-tls-certs\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.483130 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.483107 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-model-cache\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.483175 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.483137 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-home\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.483219 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.483183 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-tmp-dir\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.485012 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.484987 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-dshm\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.485415 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.485395 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98616b96-63db-43fa-82bf-7839bbf063b5-tls-certs\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.490657 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.490634 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw9tq\" (UniqueName: \"kubernetes.io/projected/98616b96-63db-43fa-82bf-7839bbf063b5-kube-api-access-hw9tq\") pod \"e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.654014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.653924 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:49:57.793045 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:57.793014 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d"] Jul 08 17:49:57.795623 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:49:57.795567 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98616b96_63db_43fa_82bf_7839bbf063b5.slice/crio-9e92271a9090cc8583f3c704864a81d4fb2c546154606c3fd1f178f78099ab82 WatchSource:0}: Error finding container 9e92271a9090cc8583f3c704864a81d4fb2c546154606c3fd1f178f78099ab82: Status 404 returned error can't find the container with id 9e92271a9090cc8583f3c704864a81d4fb2c546154606c3fd1f178f78099ab82 Jul 08 17:49:58.644242 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:58.644199 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" event={"ID":"98616b96-63db-43fa-82bf-7839bbf063b5","Type":"ContainerStarted","Data":"698b4ae6305d447e7b9e533a6c227fd142d198784833f25a1466696cb3001ab4"} Jul 08 17:49:58.644242 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:58.644246 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" event={"ID":"98616b96-63db-43fa-82bf-7839bbf063b5","Type":"ContainerStarted","Data":"9e92271a9090cc8583f3c704864a81d4fb2c546154606c3fd1f178f78099ab82"} Jul 08 17:49:58.678695 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:49:58.678639 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" podStartSLOduration=1.6786230629999999 podStartE2EDuration="1.678623063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:49:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:49:58.677023461 +0000 UTC m=+2032.903005616" watchObservedRunningTime="2026-07-08 17:49:58.678623063 +0000 UTC m=+2032.904605186" Jul 08 17:50:00.937518 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:00.937483 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:50:07.654645 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:07.654580 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:50:07.655143 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:07.654663 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:50:07.667149 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:07.667121 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:50:07.703869 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:07.703831 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:50:15.559957 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:15.559915 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:50:26.333181 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:26.333141 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:50:28.853930 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:28.853826 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d"] Jul 08 17:50:43.875077 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:43.875014 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" podUID="98616b96-63db-43fa-82bf-7839bbf063b5" containerName="main" containerID="cri-o://698b4ae6305d447e7b9e533a6c227fd142d198784833f25a1466696cb3001ab4" gracePeriod=45 Jul 08 17:50:44.130220 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.130149 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:50:44.296765 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.296701 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw9tq\" (UniqueName: \"kubernetes.io/projected/98616b96-63db-43fa-82bf-7839bbf063b5-kube-api-access-hw9tq\") pod \"98616b96-63db-43fa-82bf-7839bbf063b5\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " Jul 08 17:50:44.296765 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.296774 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-dshm\") pod \"98616b96-63db-43fa-82bf-7839bbf063b5\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " Jul 08 17:50:44.296990 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.296803 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98616b96-63db-43fa-82bf-7839bbf063b5-tls-certs\") pod \"98616b96-63db-43fa-82bf-7839bbf063b5\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " Jul 08 17:50:44.296990 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.296904 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-home\") pod \"98616b96-63db-43fa-82bf-7839bbf063b5\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " Jul 08 17:50:44.296990 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.296960 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-model-cache\") pod \"98616b96-63db-43fa-82bf-7839bbf063b5\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " Jul 08 17:50:44.297208 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.297021 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-tmp-dir\") pod \"98616b96-63db-43fa-82bf-7839bbf063b5\" (UID: \"98616b96-63db-43fa-82bf-7839bbf063b5\") " Jul 08 17:50:44.297208 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.297160 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-home" (OuterVolumeSpecName: "home") pod "98616b96-63db-43fa-82bf-7839bbf063b5" (UID: "98616b96-63db-43fa-82bf-7839bbf063b5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:50:44.297312 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.297219 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-model-cache" (OuterVolumeSpecName: "model-cache") pod "98616b96-63db-43fa-82bf-7839bbf063b5" (UID: "98616b96-63db-43fa-82bf-7839bbf063b5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:50:44.297361 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.297299 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "98616b96-63db-43fa-82bf-7839bbf063b5" (UID: "98616b96-63db-43fa-82bf-7839bbf063b5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:50:44.297419 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.297376 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-home\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:50:44.297419 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.297395 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-model-cache\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:50:44.299098 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.299078 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-dshm" (OuterVolumeSpecName: "dshm") pod "98616b96-63db-43fa-82bf-7839bbf063b5" (UID: "98616b96-63db-43fa-82bf-7839bbf063b5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:50:44.299195 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.299130 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98616b96-63db-43fa-82bf-7839bbf063b5-kube-api-access-hw9tq" (OuterVolumeSpecName: "kube-api-access-hw9tq") pod "98616b96-63db-43fa-82bf-7839bbf063b5" (UID: "98616b96-63db-43fa-82bf-7839bbf063b5"). InnerVolumeSpecName "kube-api-access-hw9tq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:50:44.299195 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.299151 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98616b96-63db-43fa-82bf-7839bbf063b5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "98616b96-63db-43fa-82bf-7839bbf063b5" (UID: "98616b96-63db-43fa-82bf-7839bbf063b5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:50:44.398041 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.397953 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98616b96-63db-43fa-82bf-7839bbf063b5-tls-certs\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:50:44.398041 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.397983 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-tmp-dir\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:50:44.398041 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.397992 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw9tq\" (UniqueName: \"kubernetes.io/projected/98616b96-63db-43fa-82bf-7839bbf063b5-kube-api-access-hw9tq\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:50:44.398041 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.398001 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98616b96-63db-43fa-82bf-7839bbf063b5-dshm\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:50:44.890292 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.890253 2578 generic.go:356] "Generic (PLEG): container finished" podID="98616b96-63db-43fa-82bf-7839bbf063b5" containerID="698b4ae6305d447e7b9e533a6c227fd142d198784833f25a1466696cb3001ab4" exitCode=0 Jul 08 17:50:44.890292 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.890294 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" Jul 08 17:50:44.890829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.890295 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" event={"ID":"98616b96-63db-43fa-82bf-7839bbf063b5","Type":"ContainerDied","Data":"698b4ae6305d447e7b9e533a6c227fd142d198784833f25a1466696cb3001ab4"} Jul 08 17:50:44.890829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.890417 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d" event={"ID":"98616b96-63db-43fa-82bf-7839bbf063b5","Type":"ContainerDied","Data":"9e92271a9090cc8583f3c704864a81d4fb2c546154606c3fd1f178f78099ab82"} Jul 08 17:50:44.890829 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.890440 2578 scope.go:117] "RemoveContainer" containerID="698b4ae6305d447e7b9e533a6c227fd142d198784833f25a1466696cb3001ab4" Jul 08 17:50:44.906045 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.906016 2578 scope.go:117] "RemoveContainer" containerID="698b4ae6305d447e7b9e533a6c227fd142d198784833f25a1466696cb3001ab4" Jul 08 17:50:44.906447 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:50:44.906417 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"698b4ae6305d447e7b9e533a6c227fd142d198784833f25a1466696cb3001ab4\": container with ID starting with 698b4ae6305d447e7b9e533a6c227fd142d198784833f25a1466696cb3001ab4 not found: ID does not exist" containerID="698b4ae6305d447e7b9e533a6c227fd142d198784833f25a1466696cb3001ab4" Jul 08 17:50:44.906520 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.906462 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"698b4ae6305d447e7b9e533a6c227fd142d198784833f25a1466696cb3001ab4"} err="failed to get container status \"698b4ae6305d447e7b9e533a6c227fd142d198784833f25a1466696cb3001ab4\": rpc error: code = NotFound desc = could not find container \"698b4ae6305d447e7b9e533a6c227fd142d198784833f25a1466696cb3001ab4\": container with ID starting with 698b4ae6305d447e7b9e533a6c227fd142d198784833f25a1466696cb3001ab4 not found: ID does not exist" Jul 08 17:50:44.917569 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.917532 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d"] Jul 08 17:50:44.920165 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:44.920136 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-525288bb/e2e-lifecycle-model-525288bb-kserve-f5c99ccc9-k4j6d"] Jul 08 17:50:46.394396 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:50:46.394364 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98616b96-63db-43fa-82bf-7839bbf063b5" path="/var/lib/kubelet/pods/98616b96-63db-43fa-82bf-7839bbf063b5/volumes" Jul 08 17:51:08.346337 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:51:08.346297 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:51:21.440775 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:51:21.440738 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:51:31.239052 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:51:31.239011 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:52:17.459729 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:17.459687 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:52:27.070036 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.070001 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls"] Jul 08 17:52:27.070420 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.070321 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98616b96-63db-43fa-82bf-7839bbf063b5" containerName="main" Jul 08 17:52:27.070420 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.070331 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="98616b96-63db-43fa-82bf-7839bbf063b5" containerName="main" Jul 08 17:52:27.070420 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.070403 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="98616b96-63db-43fa-82bf-7839bbf063b5" containerName="main" Jul 08 17:52:27.073428 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.073414 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.076760 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.076737 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a1ffe7\"/\"default-dockercfg-wp6t7\"" Jul 08 17:52:27.076760 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.076751 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a1ffe7\"/\"kube-root-ca.crt\"" Jul 08 17:52:27.076954 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.076803 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a1ffe7\"/\"test-model-a1ffe7-kserve-self-signed-certs\"" Jul 08 17:52:27.077011 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.077001 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a1ffe7\"/\"openshift-service-ca.crt\"" Jul 08 17:52:27.082954 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.082934 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls"] Jul 08 17:52:27.198768 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.198739 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-home\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.198768 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.198770 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-tmp-dir\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.198958 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.198792 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-model-cache\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.198958 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.198810 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c7bp\" (UniqueName: \"kubernetes.io/projected/f7c1aa1c-6d88-47fd-902f-84503d211e5c-kube-api-access-5c7bp\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.198958 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.198909 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-dshm\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.198958 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.198941 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7c1aa1c-6d88-47fd-902f-84503d211e5c-tls-certs\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.300232 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.300206 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-dshm\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.300375 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.300238 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7c1aa1c-6d88-47fd-902f-84503d211e5c-tls-certs\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.300375 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.300305 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-home\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.300375 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.300328 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-tmp-dir\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.300521 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.300453 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-model-cache\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.300521 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.300495 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c7bp\" (UniqueName: \"kubernetes.io/projected/f7c1aa1c-6d88-47fd-902f-84503d211e5c-kube-api-access-5c7bp\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.300687 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.300670 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-tmp-dir\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.300748 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.300733 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-home\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.300800 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.300759 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-model-cache\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.302418 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.302392 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-dshm\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.302556 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.302541 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7c1aa1c-6d88-47fd-902f-84503d211e5c-tls-certs\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.308662 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.308646 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c7bp\" (UniqueName: \"kubernetes.io/projected/f7c1aa1c-6d88-47fd-902f-84503d211e5c-kube-api-access-5c7bp\") pod \"test-model-a1ffe7-kserve-79db544f45-xnpls\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.383778 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.383705 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:27.511784 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:27.511756 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls"] Jul 08 17:52:27.514228 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:52:27.514187 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf7c1aa1c_6d88_47fd_902f_84503d211e5c.slice/crio-963cef637a7059ac895e1993960bea3db32077926389ce03467f71afcff04692 WatchSource:0}: Error finding container 963cef637a7059ac895e1993960bea3db32077926389ce03467f71afcff04692: Status 404 returned error can't find the container with id 963cef637a7059ac895e1993960bea3db32077926389ce03467f71afcff04692 Jul 08 17:52:28.424241 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:28.424208 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" event={"ID":"f7c1aa1c-6d88-47fd-902f-84503d211e5c","Type":"ContainerStarted","Data":"912a83d54a3e0e6a7e3097aa330ecd4ef6606e3e85d690a1d4736adc926bf9d9"} Jul 08 17:52:28.424689 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:28.424252 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" event={"ID":"f7c1aa1c-6d88-47fd-902f-84503d211e5c","Type":"ContainerStarted","Data":"963cef637a7059ac895e1993960bea3db32077926389ce03467f71afcff04692"} Jul 08 17:52:28.444414 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:28.444369 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" podStartSLOduration=1.44435516 podStartE2EDuration="1.44435516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:52:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:52:28.4419056 +0000 UTC m=+2182.667887711" watchObservedRunningTime="2026-07-08 17:52:28.44435516 +0000 UTC m=+2182.670337279" Jul 08 17:52:32.241932 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:32.241896 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:52:37.383906 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:37.383870 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:37.383906 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:37.383915 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:37.396508 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:37.396478 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:37.482400 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:37.482369 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:52:59.427249 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.427213 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf"] Jul 08 17:52:59.430669 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.430653 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.433810 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.433765 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-446c79\"/\"test-model-446c79-kserve-self-signed-certs\"" Jul 08 17:52:59.434756 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.434727 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-446c79\"/\"openshift-service-ca.crt\"" Jul 08 17:52:59.434863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.434784 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-446c79\"/\"default-dockercfg-9l7tj\"" Jul 08 17:52:59.434863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.434785 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-446c79\"/\"kube-root-ca.crt\"" Jul 08 17:52:59.444902 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.444874 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf"] Jul 08 17:52:59.459898 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.459871 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-tmp-dir\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.460000 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.459922 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-model-cache\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.460000 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.459969 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-dshm\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.460122 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.460105 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-home\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.460199 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.460186 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-tls-certs\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.460257 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.460216 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xts6f\" (UniqueName: \"kubernetes.io/projected/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-kube-api-access-xts6f\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.560903 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.560873 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-home\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.561073 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.560910 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-tls-certs\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.561073 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.560930 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xts6f\" (UniqueName: \"kubernetes.io/projected/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-kube-api-access-xts6f\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.561073 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.560957 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-tmp-dir\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.561073 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.560981 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-model-cache\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.561073 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.560998 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-dshm\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.561304 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.561275 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-home\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.561365 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.561344 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-tmp-dir\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.561421 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.561401 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-model-cache\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.563245 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.563221 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-dshm\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.563420 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.563401 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-tls-certs\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.569639 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.569615 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xts6f\" (UniqueName: \"kubernetes.io/projected/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-kube-api-access-xts6f\") pod \"test-model-446c79-kserve-64dc4c784c-6x2cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.740909 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.740879 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:52:59.879297 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:52:59.879260 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ebd80b9_6150_4fe2_9965_39a04e9ed1cf.slice/crio-321299b03d1fd8db951afcb6f2781ac635e837e759b1b5169ad68a71eee15536 WatchSource:0}: Error finding container 321299b03d1fd8db951afcb6f2781ac635e837e759b1b5169ad68a71eee15536: Status 404 returned error can't find the container with id 321299b03d1fd8db951afcb6f2781ac635e837e759b1b5169ad68a71eee15536 Jul 08 17:52:59.883859 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:52:59.883834 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf"] Jul 08 17:53:00.587599 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:00.587558 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" event={"ID":"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf","Type":"ContainerStarted","Data":"6c21a88962b66ed4a6f437f536514326e08c17289c8a9b40c45f0f1c807e97e9"} Jul 08 17:53:00.587967 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:00.587611 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" event={"ID":"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf","Type":"ContainerStarted","Data":"321299b03d1fd8db951afcb6f2781ac635e837e759b1b5169ad68a71eee15536"} Jul 08 17:53:00.607855 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:00.607808 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" podStartSLOduration=1.607793902 podStartE2EDuration="1.607793902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:52:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:53:00.606767784 +0000 UTC m=+2214.832749905" watchObservedRunningTime="2026-07-08 17:53:00.607793902 +0000 UTC m=+2214.833776056" Jul 08 17:53:03.653425 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:03.653389 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:53:09.741939 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:09.741905 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:53:09.742314 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:09.741964 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:53:09.754703 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:09.754680 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:53:10.655060 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:10.655023 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:53:17.481425 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:17.481388 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:53:20.881241 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:20.881206 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls"] Jul 08 17:53:22.037755 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:22.037724 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf"] Jul 08 17:53:34.561522 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:34.561446 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:53:35.901694 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:35.901629 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" podUID="f7c1aa1c-6d88-47fd-902f-84503d211e5c" containerName="main" containerID="cri-o://912a83d54a3e0e6a7e3097aa330ecd4ef6606e3e85d690a1d4736adc926bf9d9" gracePeriod=45 Jul 08 17:53:36.160304 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.160241 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:53:36.255716 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.255680 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-dshm\") pod \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " Jul 08 17:53:36.255716 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.255725 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-model-cache\") pod \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " Jul 08 17:53:36.255965 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.255750 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-tmp-dir\") pod \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " Jul 08 17:53:36.255965 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.255809 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7c1aa1c-6d88-47fd-902f-84503d211e5c-tls-certs\") pod \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " Jul 08 17:53:36.255965 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.255850 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-home\") pod \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " Jul 08 17:53:36.255965 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.255870 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5c7bp\" (UniqueName: \"kubernetes.io/projected/f7c1aa1c-6d88-47fd-902f-84503d211e5c-kube-api-access-5c7bp\") pod \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\" (UID: \"f7c1aa1c-6d88-47fd-902f-84503d211e5c\") " Jul 08 17:53:36.256169 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.256012 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-model-cache" (OuterVolumeSpecName: "model-cache") pod "f7c1aa1c-6d88-47fd-902f-84503d211e5c" (UID: "f7c1aa1c-6d88-47fd-902f-84503d211e5c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:53:36.256169 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.256046 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f7c1aa1c-6d88-47fd-902f-84503d211e5c" (UID: "f7c1aa1c-6d88-47fd-902f-84503d211e5c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:53:36.256169 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.256110 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-model-cache\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:53:36.256169 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.256128 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-home" (OuterVolumeSpecName: "home") pod "f7c1aa1c-6d88-47fd-902f-84503d211e5c" (UID: "f7c1aa1c-6d88-47fd-902f-84503d211e5c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:53:36.258024 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.258003 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-dshm" (OuterVolumeSpecName: "dshm") pod "f7c1aa1c-6d88-47fd-902f-84503d211e5c" (UID: "f7c1aa1c-6d88-47fd-902f-84503d211e5c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:53:36.258128 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.258076 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7c1aa1c-6d88-47fd-902f-84503d211e5c-kube-api-access-5c7bp" (OuterVolumeSpecName: "kube-api-access-5c7bp") pod "f7c1aa1c-6d88-47fd-902f-84503d211e5c" (UID: "f7c1aa1c-6d88-47fd-902f-84503d211e5c"). InnerVolumeSpecName "kube-api-access-5c7bp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:53:36.258128 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.258083 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7c1aa1c-6d88-47fd-902f-84503d211e5c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f7c1aa1c-6d88-47fd-902f-84503d211e5c" (UID: "f7c1aa1c-6d88-47fd-902f-84503d211e5c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:53:36.357168 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.357128 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-home\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:53:36.357168 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.357158 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5c7bp\" (UniqueName: \"kubernetes.io/projected/f7c1aa1c-6d88-47fd-902f-84503d211e5c-kube-api-access-5c7bp\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:53:36.357168 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.357172 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-dshm\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:53:36.357411 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.357181 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7c1aa1c-6d88-47fd-902f-84503d211e5c-tmp-dir\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:53:36.357411 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.357190 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7c1aa1c-6d88-47fd-902f-84503d211e5c-tls-certs\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:53:36.773826 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.773793 2578 generic.go:356] "Generic (PLEG): container finished" podID="f7c1aa1c-6d88-47fd-902f-84503d211e5c" containerID="912a83d54a3e0e6a7e3097aa330ecd4ef6606e3e85d690a1d4736adc926bf9d9" exitCode=0 Jul 08 17:53:36.773992 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.773834 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" Jul 08 17:53:36.773992 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.773863 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" event={"ID":"f7c1aa1c-6d88-47fd-902f-84503d211e5c","Type":"ContainerDied","Data":"912a83d54a3e0e6a7e3097aa330ecd4ef6606e3e85d690a1d4736adc926bf9d9"} Jul 08 17:53:36.773992 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.773895 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls" event={"ID":"f7c1aa1c-6d88-47fd-902f-84503d211e5c","Type":"ContainerDied","Data":"963cef637a7059ac895e1993960bea3db32077926389ce03467f71afcff04692"} Jul 08 17:53:36.773992 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.773914 2578 scope.go:117] "RemoveContainer" containerID="912a83d54a3e0e6a7e3097aa330ecd4ef6606e3e85d690a1d4736adc926bf9d9" Jul 08 17:53:36.788836 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.788813 2578 scope.go:117] "RemoveContainer" containerID="912a83d54a3e0e6a7e3097aa330ecd4ef6606e3e85d690a1d4736adc926bf9d9" Jul 08 17:53:36.789123 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:53:36.789103 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"912a83d54a3e0e6a7e3097aa330ecd4ef6606e3e85d690a1d4736adc926bf9d9\": container with ID starting with 912a83d54a3e0e6a7e3097aa330ecd4ef6606e3e85d690a1d4736adc926bf9d9 not found: ID does not exist" containerID="912a83d54a3e0e6a7e3097aa330ecd4ef6606e3e85d690a1d4736adc926bf9d9" Jul 08 17:53:36.789209 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.789128 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"912a83d54a3e0e6a7e3097aa330ecd4ef6606e3e85d690a1d4736adc926bf9d9"} err="failed to get container status \"912a83d54a3e0e6a7e3097aa330ecd4ef6606e3e85d690a1d4736adc926bf9d9\": rpc error: code = NotFound desc = could not find container \"912a83d54a3e0e6a7e3097aa330ecd4ef6606e3e85d690a1d4736adc926bf9d9\": container with ID starting with 912a83d54a3e0e6a7e3097aa330ecd4ef6606e3e85d690a1d4736adc926bf9d9 not found: ID does not exist" Jul 08 17:53:36.800307 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.800277 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls"] Jul 08 17:53:36.802233 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:36.802215 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-a1ffe7/test-model-a1ffe7-kserve-79db544f45-xnpls"] Jul 08 17:53:37.062562 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.062462 2578 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" podUID="1ebd80b9-6150-4fe2-9965-39a04e9ed1cf" containerName="main" containerID="cri-o://6c21a88962b66ed4a6f437f536514326e08c17289c8a9b40c45f0f1c807e97e9" gracePeriod=45 Jul 08 17:53:37.307014 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.306988 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:53:37.465943 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.465911 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-tmp-dir\") pod \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " Jul 08 17:53:37.466091 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.465953 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-home\") pod \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " Jul 08 17:53:37.466091 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.465996 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-tls-certs\") pod \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " Jul 08 17:53:37.466091 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.466047 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-model-cache\") pod \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " Jul 08 17:53:37.466091 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.466065 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xts6f\" (UniqueName: \"kubernetes.io/projected/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-kube-api-access-xts6f\") pod \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " Jul 08 17:53:37.466316 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.466092 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-dshm\") pod \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\" (UID: \"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf\") " Jul 08 17:53:37.466316 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.466235 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1ebd80b9-6150-4fe2-9965-39a04e9ed1cf" (UID: "1ebd80b9-6150-4fe2-9965-39a04e9ed1cf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:53:37.466316 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.466252 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-home" (OuterVolumeSpecName: "home") pod "1ebd80b9-6150-4fe2-9965-39a04e9ed1cf" (UID: "1ebd80b9-6150-4fe2-9965-39a04e9ed1cf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:53:37.466434 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.466328 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-tmp-dir\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:53:37.466434 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.466331 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-model-cache" (OuterVolumeSpecName: "model-cache") pod "1ebd80b9-6150-4fe2-9965-39a04e9ed1cf" (UID: "1ebd80b9-6150-4fe2-9965-39a04e9ed1cf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:53:37.466434 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.466346 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-home\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:53:37.468404 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.468367 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-dshm" (OuterVolumeSpecName: "dshm") pod "1ebd80b9-6150-4fe2-9965-39a04e9ed1cf" (UID: "1ebd80b9-6150-4fe2-9965-39a04e9ed1cf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:53:37.468509 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.468420 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1ebd80b9-6150-4fe2-9965-39a04e9ed1cf" (UID: "1ebd80b9-6150-4fe2-9965-39a04e9ed1cf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:53:37.468509 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.468435 2578 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-kube-api-access-xts6f" (OuterVolumeSpecName: "kube-api-access-xts6f") pod "1ebd80b9-6150-4fe2-9965-39a04e9ed1cf" (UID: "1ebd80b9-6150-4fe2-9965-39a04e9ed1cf"). InnerVolumeSpecName "kube-api-access-xts6f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:53:37.567284 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.567254 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-model-cache\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:53:37.567284 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.567279 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xts6f\" (UniqueName: \"kubernetes.io/projected/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-kube-api-access-xts6f\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:53:37.567284 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.567289 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-dshm\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:53:37.567512 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.567298 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf-tls-certs\") on node \"ip-10-0-128-50.ec2.internal\" DevicePath \"\"" Jul 08 17:53:37.780700 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.780609 2578 generic.go:356] "Generic (PLEG): container finished" podID="1ebd80b9-6150-4fe2-9965-39a04e9ed1cf" containerID="6c21a88962b66ed4a6f437f536514326e08c17289c8a9b40c45f0f1c807e97e9" exitCode=0 Jul 08 17:53:37.780700 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.780655 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" Jul 08 17:53:37.780881 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.780695 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" event={"ID":"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf","Type":"ContainerDied","Data":"6c21a88962b66ed4a6f437f536514326e08c17289c8a9b40c45f0f1c807e97e9"} Jul 08 17:53:37.780881 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.780738 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf" event={"ID":"1ebd80b9-6150-4fe2-9965-39a04e9ed1cf","Type":"ContainerDied","Data":"321299b03d1fd8db951afcb6f2781ac635e837e759b1b5169ad68a71eee15536"} Jul 08 17:53:37.780881 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.780755 2578 scope.go:117] "RemoveContainer" containerID="6c21a88962b66ed4a6f437f536514326e08c17289c8a9b40c45f0f1c807e97e9" Jul 08 17:53:37.790638 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.790619 2578 scope.go:117] "RemoveContainer" containerID="6c21a88962b66ed4a6f437f536514326e08c17289c8a9b40c45f0f1c807e97e9" Jul 08 17:53:37.790894 ip-10-0-128-50 kubenswrapper[2578]: E0708 17:53:37.790874 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c21a88962b66ed4a6f437f536514326e08c17289c8a9b40c45f0f1c807e97e9\": container with ID starting with 6c21a88962b66ed4a6f437f536514326e08c17289c8a9b40c45f0f1c807e97e9 not found: ID does not exist" containerID="6c21a88962b66ed4a6f437f536514326e08c17289c8a9b40c45f0f1c807e97e9" Jul 08 17:53:37.790961 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.790903 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c21a88962b66ed4a6f437f536514326e08c17289c8a9b40c45f0f1c807e97e9"} err="failed to get container status \"6c21a88962b66ed4a6f437f536514326e08c17289c8a9b40c45f0f1c807e97e9\": rpc error: code = NotFound desc = could not find container \"6c21a88962b66ed4a6f437f536514326e08c17289c8a9b40c45f0f1c807e97e9\": container with ID starting with 6c21a88962b66ed4a6f437f536514326e08c17289c8a9b40c45f0f1c807e97e9 not found: ID does not exist" Jul 08 17:53:37.808940 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.808912 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf"] Jul 08 17:53:37.813834 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:37.813812 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-446c79/test-model-446c79-kserve-64dc4c784c-6x2cf"] Jul 08 17:53:38.393623 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:38.393576 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ebd80b9-6150-4fe2-9965-39a04e9ed1cf" path="/var/lib/kubelet/pods/1ebd80b9-6150-4fe2-9965-39a04e9ed1cf/volumes" Jul 08 17:53:38.394027 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:53:38.394012 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7c1aa1c-6d88-47fd-902f-84503d211e5c" path="/var/lib/kubelet/pods/f7c1aa1c-6d88-47fd-902f-84503d211e5c/volumes" Jul 08 17:54:10.162928 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:54:10.162891 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:54:21.344455 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:54:21.344414 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:54:31.055600 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:54:31.055545 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:54:47.442901 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:54:47.442863 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:54:59.244511 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:54:59.244409 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:55:15.740157 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:55:15.740110 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:58:16.148001 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:16.147921 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:58:26.419447 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:26.419416 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57766c5757-zt6pp_ad2ee5ab-dceb-4b51-9547-5a05f443e29c/authorino/0.log" Jul 08 17:58:28.448287 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:28.448240 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-52pw8"] Jul 08 17:58:32.151283 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:32.151255 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-564755f94b-2djl5_2c051e1c-f006-4118-a7cd-6825f368f632/copy-manifests/0.log" Jul 08 17:58:32.164090 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:32.164068 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-564755f94b-2djl5_2c051e1c-f006-4118-a7cd-6825f368f632/manager/0.log" Jul 08 17:58:33.321118 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:33.321086 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl_ae794944-014d-49fc-84e4-4af22f676baf/util/0.log" Jul 08 17:58:33.329718 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:33.329692 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl_ae794944-014d-49fc-84e4-4af22f676baf/pull/0.log" Jul 08 17:58:33.336271 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:33.336251 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl_ae794944-014d-49fc-84e4-4af22f676baf/extract/0.log" Jul 08 17:58:33.458895 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:33.458876 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h_27a036b2-a643-4d1b-a38d-9b86bd249d71/util/0.log" Jul 08 17:58:33.465420 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:33.465402 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h_27a036b2-a643-4d1b-a38d-9b86bd249d71/pull/0.log" Jul 08 17:58:33.471644 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:33.471626 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h_27a036b2-a643-4d1b-a38d-9b86bd249d71/extract/0.log" Jul 08 17:58:33.593925 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:33.593860 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t_3f041301-1c23-44be-ab8e-1cd9d6f0f62a/extract/0.log" Jul 08 17:58:33.608944 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:33.608928 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t_3f041301-1c23-44be-ab8e-1cd9d6f0f62a/util/0.log" Jul 08 17:58:33.626267 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:33.626248 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t_3f041301-1c23-44be-ab8e-1cd9d6f0f62a/pull/0.log" Jul 08 17:58:33.761466 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:33.761446 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq_d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb/util/0.log" Jul 08 17:58:33.771640 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:33.771618 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq_d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb/pull/0.log" Jul 08 17:58:33.778246 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:33.778230 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq_d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb/extract/0.log" Jul 08 17:58:33.907176 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:33.907122 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57766c5757-zt6pp_ad2ee5ab-dceb-4b51-9547-5a05f443e29c/authorino/0.log" Jul 08 17:58:34.294385 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:34.294365 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9rx8v_1b813e7d-1e02-4c29-b90b-d9f4f6eb910c/kuadrant-console-plugin/0.log" Jul 08 17:58:34.699497 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:34.699473 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-52pw8_9a2dfb66-3220-4b55-8a32-51deec52ac06/limitador/0.log" Jul 08 17:58:35.596309 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:35.596283 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d86b88db9-rvndc_7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f/kube-auth-proxy/0.log" Jul 08 17:58:36.115157 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:36.115130 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79f98bd59-6bd9x_ed90d90c-0640-40ee-9e52-ad99c028002a/router/0.log" Jul 08 17:58:37.098229 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:37.098205 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-xrbg7_184f0973-0787-4ff2-bcc0-cb08c24a6fda/main/0.log" Jul 08 17:58:41.675641 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.675611 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ksr2v/must-gather-9fw6k"] Jul 08 17:58:41.676001 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.675909 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c1aa1c-6d88-47fd-902f-84503d211e5c" containerName="main" Jul 08 17:58:41.676001 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.675920 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c1aa1c-6d88-47fd-902f-84503d211e5c" containerName="main" Jul 08 17:58:41.676001 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.675930 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ebd80b9-6150-4fe2-9965-39a04e9ed1cf" containerName="main" Jul 08 17:58:41.676001 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.675936 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ebd80b9-6150-4fe2-9965-39a04e9ed1cf" containerName="main" Jul 08 17:58:41.676001 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.676001 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7c1aa1c-6d88-47fd-902f-84503d211e5c" containerName="main" Jul 08 17:58:41.676162 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.676008 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ebd80b9-6150-4fe2-9965-39a04e9ed1cf" containerName="main" Jul 08 17:58:41.678856 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.678833 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ksr2v/must-gather-9fw6k" Jul 08 17:58:41.681170 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.681150 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ksr2v\"/\"kube-root-ca.crt\"" Jul 08 17:58:41.681966 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.681932 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ksr2v\"/\"default-dockercfg-wqkzb\"" Jul 08 17:58:41.682029 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.681932 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ksr2v\"/\"openshift-service-ca.crt\"" Jul 08 17:58:41.691863 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.691837 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ksr2v/must-gather-9fw6k"] Jul 08 17:58:41.825188 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.825161 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/849760f0-9438-4d47-aea4-fd0fdb05427b-must-gather-output\") pod \"must-gather-9fw6k\" (UID: \"849760f0-9438-4d47-aea4-fd0fdb05427b\") " pod="openshift-must-gather-ksr2v/must-gather-9fw6k" Jul 08 17:58:41.825188 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.825190 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2vxv\" (UniqueName: \"kubernetes.io/projected/849760f0-9438-4d47-aea4-fd0fdb05427b-kube-api-access-x2vxv\") pod \"must-gather-9fw6k\" (UID: \"849760f0-9438-4d47-aea4-fd0fdb05427b\") " pod="openshift-must-gather-ksr2v/must-gather-9fw6k" Jul 08 17:58:41.925981 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.925900 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/849760f0-9438-4d47-aea4-fd0fdb05427b-must-gather-output\") pod \"must-gather-9fw6k\" (UID: \"849760f0-9438-4d47-aea4-fd0fdb05427b\") " pod="openshift-must-gather-ksr2v/must-gather-9fw6k" Jul 08 17:58:41.925981 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.925938 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2vxv\" (UniqueName: \"kubernetes.io/projected/849760f0-9438-4d47-aea4-fd0fdb05427b-kube-api-access-x2vxv\") pod \"must-gather-9fw6k\" (UID: \"849760f0-9438-4d47-aea4-fd0fdb05427b\") " pod="openshift-must-gather-ksr2v/must-gather-9fw6k" Jul 08 17:58:41.926231 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.926213 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/849760f0-9438-4d47-aea4-fd0fdb05427b-must-gather-output\") pod \"must-gather-9fw6k\" (UID: \"849760f0-9438-4d47-aea4-fd0fdb05427b\") " pod="openshift-must-gather-ksr2v/must-gather-9fw6k" Jul 08 17:58:41.933775 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.933754 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2vxv\" (UniqueName: \"kubernetes.io/projected/849760f0-9438-4d47-aea4-fd0fdb05427b-kube-api-access-x2vxv\") pod \"must-gather-9fw6k\" (UID: \"849760f0-9438-4d47-aea4-fd0fdb05427b\") " pod="openshift-must-gather-ksr2v/must-gather-9fw6k" Jul 08 17:58:41.986786 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:41.986765 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ksr2v/must-gather-9fw6k" Jul 08 17:58:42.112671 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:42.112643 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ksr2v/must-gather-9fw6k"] Jul 08 17:58:42.114723 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:58:42.114694 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod849760f0_9438_4d47_aea4_fd0fdb05427b.slice/crio-c33cae468cbf08aaba5d24775e29e21c0cb5e9a0299df88d9b24591a0e041d69 WatchSource:0}: Error finding container c33cae468cbf08aaba5d24775e29e21c0cb5e9a0299df88d9b24591a0e041d69: Status 404 returned error can't find the container with id c33cae468cbf08aaba5d24775e29e21c0cb5e9a0299df88d9b24591a0e041d69 Jul 08 17:58:42.116479 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:42.116461 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 17:58:42.326488 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:42.326461 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ksr2v/must-gather-9fw6k" event={"ID":"849760f0-9438-4d47-aea4-fd0fdb05427b","Type":"ContainerStarted","Data":"c33cae468cbf08aaba5d24775e29e21c0cb5e9a0299df88d9b24591a0e041d69"} Jul 08 17:58:43.334594 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:43.334544 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ksr2v/must-gather-9fw6k" event={"ID":"849760f0-9438-4d47-aea4-fd0fdb05427b","Type":"ContainerStarted","Data":"d4a12f3eaf2910ac66f8968a91f2a3b4d6363c67fb7ba45793971bc71f2b8b01"} Jul 08 17:58:44.341539 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:44.341507 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ksr2v/must-gather-9fw6k" event={"ID":"849760f0-9438-4d47-aea4-fd0fdb05427b","Type":"ContainerStarted","Data":"f7f6f70d0d1edce4d8f5cddd31c1a63daf012dc89fea85693f7d799dd53395f5"} Jul 08 17:58:44.358515 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:44.358455 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ksr2v/must-gather-9fw6k" podStartSLOduration=2.313156354 podStartE2EDuration="3.358433737s" totalImagesPullingTime="1.045277383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:58:44.355014672 +0000 UTC m=+2558.580996790" watchObservedRunningTime="2026-07-08 17:58:44.358433737 +0000 UTC m=+2558.584415857" Jul 08 17:58:44.738933 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:44.738887 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-pqz5t_17ebcc8b-068a-42c0-b17f-ebae02312e5b/global-pull-secret-syncer/0.log" Jul 08 17:58:44.869375 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:44.869342 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-v2p4n_9b996b12-627f-434f-bc7a-6abe5d7907e3/konnectivity-agent/0.log" Jul 08 17:58:44.920643 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:44.920608 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-50.ec2.internal_04a30fea7d8cb69c63b01d701b00e3be/haproxy/0.log" Jul 08 17:58:48.901280 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:48.901250 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl_ae794944-014d-49fc-84e4-4af22f676baf/extract/0.log" Jul 08 17:58:48.922783 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:48.922760 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl_ae794944-014d-49fc-84e4-4af22f676baf/util/0.log" Jul 08 17:58:48.944139 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:48.944113 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759fkfrl_ae794944-014d-49fc-84e4-4af22f676baf/pull/0.log" Jul 08 17:58:48.985285 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:48.985258 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h_27a036b2-a643-4d1b-a38d-9b86bd249d71/extract/0.log" Jul 08 17:58:49.007105 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:49.007082 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h_27a036b2-a643-4d1b-a38d-9b86bd249d71/util/0.log" Jul 08 17:58:49.030630 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:49.030600 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04ph4h_27a036b2-a643-4d1b-a38d-9b86bd249d71/pull/0.log" Jul 08 17:58:49.056440 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:49.056412 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t_3f041301-1c23-44be-ab8e-1cd9d6f0f62a/extract/0.log" Jul 08 17:58:49.082007 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:49.081976 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t_3f041301-1c23-44be-ab8e-1cd9d6f0f62a/util/0.log" Jul 08 17:58:49.106459 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:49.106428 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73brx4t_3f041301-1c23-44be-ab8e-1cd9d6f0f62a/pull/0.log" Jul 08 17:58:49.135972 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:49.135934 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq_d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb/extract/0.log" Jul 08 17:58:49.159320 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:49.159238 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq_d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb/util/0.log" Jul 08 17:58:49.181507 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:49.181476 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dbwcq_d7052a54-e1c0-4dd4-b8f1-ffa9a6c33ccb/pull/0.log" Jul 08 17:58:49.542961 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:49.542532 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-57766c5757-zt6pp_ad2ee5ab-dceb-4b51-9547-5a05f443e29c/authorino/0.log" Jul 08 17:58:49.652960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:49.652929 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9rx8v_1b813e7d-1e02-4c29-b90b-d9f4f6eb910c/kuadrant-console-plugin/0.log" Jul 08 17:58:49.862058 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:49.861947 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-52pw8_9a2dfb66-3220-4b55-8a32-51deec52ac06/limitador/0.log" Jul 08 17:58:51.577765 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:51.576025 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-qdcgz_21677fb3-72b7-4a6d-acf9-47f6cabde040/cluster-monitoring-operator/0.log" Jul 08 17:58:51.862198 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:51.862083 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rswcb_fd9cd1e8-2df1-4133-b1c1-54673fc66875/node-exporter/0.log" Jul 08 17:58:51.883335 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:51.883296 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rswcb_fd9cd1e8-2df1-4133-b1c1-54673fc66875/kube-rbac-proxy/0.log" Jul 08 17:58:51.902635 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:51.902576 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rswcb_fd9cd1e8-2df1-4133-b1c1-54673fc66875/init-textfile/0.log" Jul 08 17:58:52.041069 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:52.041029 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-r8spq_77dd845b-a2ed-4eb0-bb1b-30e46c4c340a/kube-rbac-proxy-main/0.log" Jul 08 17:58:52.080405 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:52.080352 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-r8spq_77dd845b-a2ed-4eb0-bb1b-30e46c4c340a/kube-rbac-proxy-self/0.log" Jul 08 17:58:52.109860 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:52.106293 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-r8spq_77dd845b-a2ed-4eb0-bb1b-30e46c4c340a/openshift-state-metrics/0.log" Jul 08 17:58:52.340954 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:52.340925 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-9jczn_0ebb1aec-031c-407f-8b34-49b723a6f2d7/prometheus-operator/0.log" Jul 08 17:58:52.360750 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:52.360711 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-9jczn_0ebb1aec-031c-407f-8b34-49b723a6f2d7/kube-rbac-proxy/0.log" Jul 08 17:58:52.422767 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:52.422714 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54f65bbbc-xtm5p_42d242fa-3adb-4f60-b06d-9d0cc9dbbd45/telemeter-client/0.log" Jul 08 17:58:52.446844 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:52.446807 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54f65bbbc-xtm5p_42d242fa-3adb-4f60-b06d-9d0cc9dbbd45/reload/0.log" Jul 08 17:58:52.468498 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:52.468422 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-54f65bbbc-xtm5p_42d242fa-3adb-4f60-b06d-9d0cc9dbbd45/kube-rbac-proxy/0.log" Jul 08 17:58:53.851811 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:53.851746 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-njbnd_626e8de7-fd53-4204-8f73-9d1d41b32d19/networking-console-plugin/0.log" Jul 08 17:58:54.676538 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.676507 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5"] Jul 08 17:58:54.682461 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.682434 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:54.688793 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.688770 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5"] Jul 08 17:58:54.856541 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.856509 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kft2c\" (UniqueName: \"kubernetes.io/projected/220570c8-2465-4e68-9b31-1748b18eb944-kube-api-access-kft2c\") pod \"perf-node-gather-daemonset-rf4c5\" (UID: \"220570c8-2465-4e68-9b31-1748b18eb944\") " pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:54.856988 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.856652 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/220570c8-2465-4e68-9b31-1748b18eb944-sys\") pod \"perf-node-gather-daemonset-rf4c5\" (UID: \"220570c8-2465-4e68-9b31-1748b18eb944\") " pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:54.856988 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.856693 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/220570c8-2465-4e68-9b31-1748b18eb944-proc\") pod \"perf-node-gather-daemonset-rf4c5\" (UID: \"220570c8-2465-4e68-9b31-1748b18eb944\") " pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:54.856988 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.856750 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/220570c8-2465-4e68-9b31-1748b18eb944-podres\") pod \"perf-node-gather-daemonset-rf4c5\" (UID: \"220570c8-2465-4e68-9b31-1748b18eb944\") " pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:54.856988 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.856876 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/220570c8-2465-4e68-9b31-1748b18eb944-lib-modules\") pod \"perf-node-gather-daemonset-rf4c5\" (UID: \"220570c8-2465-4e68-9b31-1748b18eb944\") " pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:54.957960 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.957929 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kft2c\" (UniqueName: \"kubernetes.io/projected/220570c8-2465-4e68-9b31-1748b18eb944-kube-api-access-kft2c\") pod \"perf-node-gather-daemonset-rf4c5\" (UID: \"220570c8-2465-4e68-9b31-1748b18eb944\") " pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:54.958148 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.957995 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/220570c8-2465-4e68-9b31-1748b18eb944-sys\") pod \"perf-node-gather-daemonset-rf4c5\" (UID: \"220570c8-2465-4e68-9b31-1748b18eb944\") " pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:54.958148 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.958016 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/220570c8-2465-4e68-9b31-1748b18eb944-proc\") pod \"perf-node-gather-daemonset-rf4c5\" (UID: \"220570c8-2465-4e68-9b31-1748b18eb944\") " pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:54.958148 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.958050 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/220570c8-2465-4e68-9b31-1748b18eb944-podres\") pod \"perf-node-gather-daemonset-rf4c5\" (UID: \"220570c8-2465-4e68-9b31-1748b18eb944\") " pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:54.958148 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.958083 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/220570c8-2465-4e68-9b31-1748b18eb944-lib-modules\") pod \"perf-node-gather-daemonset-rf4c5\" (UID: \"220570c8-2465-4e68-9b31-1748b18eb944\") " pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:54.958148 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.958121 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/220570c8-2465-4e68-9b31-1748b18eb944-sys\") pod \"perf-node-gather-daemonset-rf4c5\" (UID: \"220570c8-2465-4e68-9b31-1748b18eb944\") " pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:54.958385 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.958179 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/220570c8-2465-4e68-9b31-1748b18eb944-proc\") pod \"perf-node-gather-daemonset-rf4c5\" (UID: \"220570c8-2465-4e68-9b31-1748b18eb944\") " pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:54.958385 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.958222 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/220570c8-2465-4e68-9b31-1748b18eb944-lib-modules\") pod \"perf-node-gather-daemonset-rf4c5\" (UID: \"220570c8-2465-4e68-9b31-1748b18eb944\") " pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:54.958385 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.958245 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/220570c8-2465-4e68-9b31-1748b18eb944-podres\") pod \"perf-node-gather-daemonset-rf4c5\" (UID: \"220570c8-2465-4e68-9b31-1748b18eb944\") " pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:54.967345 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:54.967312 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kft2c\" (UniqueName: \"kubernetes.io/projected/220570c8-2465-4e68-9b31-1748b18eb944-kube-api-access-kft2c\") pod \"perf-node-gather-daemonset-rf4c5\" (UID: \"220570c8-2465-4e68-9b31-1748b18eb944\") " pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:55.001263 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:55.001231 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:55.178399 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:55.178356 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5"] Jul 08 17:58:55.179787 ip-10-0-128-50 kubenswrapper[2578]: W0708 17:58:55.179758 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod220570c8_2465_4e68_9b31_1748b18eb944.slice/crio-f46ca067d53e2e294724fe31668178bfca9b538068bf6e3243889bf39019f42a WatchSource:0}: Error finding container f46ca067d53e2e294724fe31668178bfca9b538068bf6e3243889bf39019f42a: Status 404 returned error can't find the container with id f46ca067d53e2e294724fe31668178bfca9b538068bf6e3243889bf39019f42a Jul 08 17:58:55.444597 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:55.444539 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" event={"ID":"220570c8-2465-4e68-9b31-1748b18eb944","Type":"ContainerStarted","Data":"1abe40e6a0d24bd61b946b3254dc877afceb036c3d96b9fb62ee0d3d61fd13f5"} Jul 08 17:58:55.444781 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:55.444631 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" event={"ID":"220570c8-2465-4e68-9b31-1748b18eb944","Type":"ContainerStarted","Data":"f46ca067d53e2e294724fe31668178bfca9b538068bf6e3243889bf39019f42a"} Jul 08 17:58:55.444781 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:55.444775 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:58:55.459999 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:55.459936 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" podStartSLOduration=1.459916762 podStartE2EDuration="1.459916762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:58:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:58:55.458539326 +0000 UTC m=+2569.684521493" watchObservedRunningTime="2026-07-08 17:58:55.459916762 +0000 UTC m=+2569.685898881" Jul 08 17:58:56.580323 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:56.580288 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mqrpr_c83a300a-7a84-40ed-85fd-baa1a196f0f3/dns/0.log" Jul 08 17:58:56.600219 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:56.600189 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mqrpr_c83a300a-7a84-40ed-85fd-baa1a196f0f3/kube-rbac-proxy/0.log" Jul 08 17:58:56.709159 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:56.709109 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xntds_abdf9cce-f248-42d1-8b24-1f03dcc28ae6/dns-node-resolver/0.log" Jul 08 17:58:57.231053 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:57.231028 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7555c_5d3f876c-9309-476f-b7ff-88c6afd11721/node-ca/0.log" Jul 08 17:58:58.354570 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:58.354539 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d86b88db9-rvndc_7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f/kube-auth-proxy/0.log" Jul 08 17:58:58.536556 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:58.536529 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79f98bd59-6bd9x_ed90d90c-0640-40ee-9e52-ad99c028002a/router/0.log" Jul 08 17:58:59.177225 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:59.177198 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7lmsl_75d2bfa3-7826-4c84-affb-b589ad1fa15e/serve-healthcheck-canary/0.log" Jul 08 17:58:59.708768 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:59.708739 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-vwfjt_77fc57aa-c23d-4c29-8529-4ca576c115b2/insights-operator/0.log" Jul 08 17:58:59.709292 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:59.709263 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-vwfjt_77fc57aa-c23d-4c29-8529-4ca576c115b2/insights-operator/1.log" Jul 08 17:58:59.886005 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:59.885975 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mdqd8_f7a07c99-280f-4bd0-97ff-c662de6cc3a5/kube-rbac-proxy/0.log" Jul 08 17:58:59.908253 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:59.908230 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mdqd8_f7a07c99-280f-4bd0-97ff-c662de6cc3a5/exporter/0.log" Jul 08 17:58:59.929855 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:58:59.929829 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mdqd8_f7a07c99-280f-4bd0-97ff-c662de6cc3a5/extractor/0.log" Jul 08 17:59:01.463851 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:01.463823 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ksr2v/perf-node-gather-daemonset-rf4c5" Jul 08 17:59:02.348230 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:02.348194 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-564755f94b-2djl5_2c051e1c-f006-4118-a7cd-6825f368f632/manager/0.log" Jul 08 17:59:02.364896 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:02.364872 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-564755f94b-2djl5_2c051e1c-f006-4118-a7cd-6825f368f632/copy-manifests/0.log" Jul 08 17:59:03.740835 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:03.740786 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8596cf9c79-4mr5q_fb2f85ac-9da4-495a-8790-e45fa0b59701/manager/0.log" Jul 08 17:59:03.868019 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:03.867932 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-wjj5t_fafa4837-ed67-4fcf-a39d-8979c76a7de3/openshift-lws-operator/0.log" Jul 08 17:59:04.704143 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:04.704117 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d86b88db9-rvndc_7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f/kube-auth-proxy/0.log" Jul 08 17:59:04.887132 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:04.887103 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79f98bd59-6bd9x_ed90d90c-0640-40ee-9e52-ad99c028002a/router/0.log" Jul 08 17:59:06.142985 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:06.142908 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6d86b88db9-rvndc_7a372ab1-23fa-4c4e-ae1b-fd5431f18f6f/kube-auth-proxy/0.log" Jul 08 17:59:06.320237 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:06.320209 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79f98bd59-6bd9x_ed90d90c-0640-40ee-9e52-ad99c028002a/router/0.log" Jul 08 17:59:09.393314 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:09.393283 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-qdcqs_5b1cbe60-67d7-4c48-866a-73760293f220/kube-storage-version-migrator-operator/1.log" Jul 08 17:59:09.393994 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:09.393975 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-qdcqs_5b1cbe60-67d7-4c48-866a-73760293f220/kube-storage-version-migrator-operator/0.log" Jul 08 17:59:10.343674 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:10.343624 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9wmcv_9adf63f6-5d10-4065-aa17-9fd6fdb55f1b/kube-multus/0.log" Jul 08 17:59:10.368204 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:10.368172 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2hjbv_f238c743-a6a4-4b86-b9ef-081f6b2ae9d1/kube-multus-additional-cni-plugins/0.log" Jul 08 17:59:10.390917 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:10.390882 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2hjbv_f238c743-a6a4-4b86-b9ef-081f6b2ae9d1/egress-router-binary-copy/0.log" Jul 08 17:59:10.413322 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:10.413302 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2hjbv_f238c743-a6a4-4b86-b9ef-081f6b2ae9d1/cni-plugins/0.log" Jul 08 17:59:10.436166 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:10.436136 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2hjbv_f238c743-a6a4-4b86-b9ef-081f6b2ae9d1/bond-cni-plugin/0.log" Jul 08 17:59:10.459875 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:10.459853 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2hjbv_f238c743-a6a4-4b86-b9ef-081f6b2ae9d1/routeoverride-cni/0.log" Jul 08 17:59:10.481293 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:10.481272 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2hjbv_f238c743-a6a4-4b86-b9ef-081f6b2ae9d1/whereabouts-cni-bincopy/0.log" Jul 08 17:59:10.504508 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:10.504484 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2hjbv_f238c743-a6a4-4b86-b9ef-081f6b2ae9d1/whereabouts-cni/0.log" Jul 08 17:59:10.904616 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:10.904550 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8dk7z_78604c5e-b53b-4761-b62e-b7f964af3e44/network-metrics-daemon/0.log" Jul 08 17:59:10.926026 ip-10-0-128-50 kubenswrapper[2578]: I0708 17:59:10.925998 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8dk7z_78604c5e-b53b-4761-b62e-b7f964af3e44/kube-rbac-proxy/0.log"