Jul 22 02:55:59.306748 ip-10-0-138-55 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 22 02:55:59.306761 ip-10-0-138-55 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 22 02:55:59.306777 ip-10-0-138-55 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 22 02:55:59.307098 ip-10-0-138-55 systemd[1]: Failed to start Kubernetes Kubelet. Jul 22 02:56:08.607607 ip-10-0-138-55 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 22 02:56:08.607618 ip-10-0-138-55 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 9b2bfca6bcb846caaed1840cb197ceca -- Jul 22 02:57:49.308799 ip-10-0-138-55 systemd[1]: Starting Kubernetes Kubelet... Jul 22 02:57:49.742758 ip-10-0-138-55 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 02:57:49.742758 ip-10-0-138-55 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 02:57:49.742758 ip-10-0-138-55 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 02:57:49.742758 ip-10-0-138-55 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 02:57:49.742758 ip-10-0-138-55 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 02:57:49.742758 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.689839 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 02:57:49.742758 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698148 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:57:49.742758 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698162 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:57:49.742758 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698165 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:57:49.742758 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698169 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:57:49.742758 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698172 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:57:49.742758 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698175 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:57:49.742758 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698177 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:57:49.742758 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698181 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698185 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698189 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698192 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698195 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698198 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698201 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698203 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698206 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698209 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698219 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698222 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698225 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698227 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698230 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698232 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698235 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698237 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698240 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:57:49.743606 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698243 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698246 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698248 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698251 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698254 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698256 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698258 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698261 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698263 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698266 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698270 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698274 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698277 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698280 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698283 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698285 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698288 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698290 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698293 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698295 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:57:49.744238 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698298 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698300 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698303 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698306 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698310 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698312 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698315 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698317 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698320 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698322 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698325 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698327 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698330 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698333 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698335 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698338 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698341 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698345 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698348 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698350 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:57:49.744843 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698353 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698357 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698360 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698362 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698365 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698367 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698370 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698372 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698375 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698377 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698380 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698382 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698385 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698387 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698390 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698392 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698395 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698398 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698401 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698403 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:57:49.745473 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698406 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698408 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698411 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698413 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698416 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698418 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698421 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698423 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698425 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698428 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698430 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698433 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.698436 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699779 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699787 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699791 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699794 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699798 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699800 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699803 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:57:49.746114 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699806 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699809 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699811 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699814 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699817 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699820 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699822 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699825 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699828 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699831 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699837 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699840 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699842 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699845 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699847 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699850 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699853 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699855 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699858 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699860 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:57:49.746770 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699863 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699866 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699868 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699871 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699873 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699876 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699878 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699882 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699884 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699887 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699890 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699894 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699897 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699900 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699903 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699906 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699909 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699912 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699915 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:57:49.747301 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699917 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699922 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699926 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699928 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699932 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699935 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699938 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699940 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699943 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699946 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699948 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699951 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699953 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699956 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699959 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699965 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699968 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699970 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699974 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:57:49.747864 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699977 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699979 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699983 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699985 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699988 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699990 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699993 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699995 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.699998 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700000 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700004 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700007 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700009 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700012 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700014 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700017 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700019 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700022 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700026 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700029 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:57:49.748366 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700032 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700035 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700037 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700040 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700042 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700045 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700047 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700050 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700054 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700056 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700059 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700063 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700066 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700068 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700448 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700453 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700456 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700459 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700462 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700464 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:57:49.748980 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700467 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700469 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700472 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700475 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700477 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700480 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700482 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700485 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700488 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700491 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700494 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700496 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700499 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700501 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700504 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700506 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700508 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700527 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700530 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700532 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:57:49.749502 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700536 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700539 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700541 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700545 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700548 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700550 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700552 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700556 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700558 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700561 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700564 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700566 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700569 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700571 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700574 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700578 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700582 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700585 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700588 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700591 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:57:49.750074 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700594 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700596 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700599 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700602 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700605 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700608 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700611 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700613 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700616 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700618 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700621 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700623 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700627 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700631 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700635 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700640 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700642 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700645 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700648 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:57:49.750596 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700651 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700654 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700656 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700659 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700662 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700664 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700667 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700669 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700671 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700674 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700676 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700679 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700681 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700684 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700686 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700689 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700692 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700695 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700697 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700700 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:57:49.751125 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700702 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700706 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700709 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700711 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700714 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700718 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700721 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700723 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700728 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700732 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700734 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700737 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700739 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.700742 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700810 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700816 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700822 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700827 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700830 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700833 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700837 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 02:57:49.751662 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700841 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700845 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700848 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700851 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700854 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700857 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700860 2580 flags.go:64] FLAG: --cgroup-root="" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700863 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700866 2580 flags.go:64] FLAG: --client-ca-file="" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700870 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700872 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700877 2580 flags.go:64] FLAG: --cluster-domain="" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700880 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700883 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700886 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700889 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700894 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700896 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700900 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700905 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700907 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700910 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700913 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700916 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 02:57:49.752388 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700919 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700923 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700926 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700929 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700932 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700935 2580 flags.go:64] FLAG: --enable-server="true" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700938 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700942 2580 flags.go:64] FLAG: --event-burst="100" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700945 2580 flags.go:64] FLAG: --event-qps="50" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700948 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700951 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700954 2580 flags.go:64] FLAG: --eviction-hard="" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700957 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700960 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700963 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700966 2580 flags.go:64] FLAG: --eviction-soft="" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700969 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700972 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700975 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700978 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700980 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700983 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700986 2580 flags.go:64] FLAG: --feature-gates="" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700989 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700994 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 02:57:49.753038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.700997 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701000 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701004 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701007 2580 flags.go:64] FLAG: --help="false" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701010 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-138-55.ec2.internal" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701013 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701016 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701019 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701022 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701025 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701028 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701031 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701034 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701037 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701039 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701042 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701045 2580 flags.go:64] FLAG: --kube-reserved="" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701048 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701051 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701053 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701056 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701059 2580 flags.go:64] FLAG: --lock-file="" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701062 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701065 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 02:57:49.753911 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701068 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701073 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701077 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701079 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701082 2580 flags.go:64] FLAG: --logging-format="text" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701085 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701088 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701092 2580 flags.go:64] FLAG: --manifest-url="" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701095 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701099 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701104 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701108 2580 flags.go:64] FLAG: --max-pods="110" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701110 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701114 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701116 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701119 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701122 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701125 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701128 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701135 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701138 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701141 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701144 2580 flags.go:64] FLAG: --pod-cidr="" Jul 22 02:57:49.758099 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701147 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701152 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701155 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701157 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701160 2580 flags.go:64] FLAG: --port="10250" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701163 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701166 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03f665b096e8b9ddc" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701169 2580 flags.go:64] FLAG: --qos-reserved="" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701172 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701175 2580 flags.go:64] FLAG: --register-node="true" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701178 2580 flags.go:64] FLAG: --register-with-taints="" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701181 2580 flags.go:64] FLAG: --registry-burst="10" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701185 2580 flags.go:64] FLAG: --registry-qps="5" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701187 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701190 2580 flags.go:64] FLAG: --reserved-memory="" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701194 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701197 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701201 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701205 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701209 2580 flags.go:64] FLAG: --runonce="false" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701212 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701214 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701217 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701220 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701223 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701226 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 02:57:49.822703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701229 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 02:57:49.771052 ip-10-0-138-55 systemd[1]: Started Kubernetes Kubelet. Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701232 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701234 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701237 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701240 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701243 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701246 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701249 2580 flags.go:64] FLAG: --system-cgroups="" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701251 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701257 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701260 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701262 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701266 2580 flags.go:64] FLAG: --tls-min-version="" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701269 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701272 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701274 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701277 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701280 2580 flags.go:64] FLAG: --v="2" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701284 2580 flags.go:64] FLAG: --version="false" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701288 2580 flags.go:64] FLAG: --vmodule="" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701292 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.701295 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701408 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701413 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:57:49.830204 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701416 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701420 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701423 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701426 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701428 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701431 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701433 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701436 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701438 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701441 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701443 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701446 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701449 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701453 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701456 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701459 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701462 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701465 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701468 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:57:49.831207 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701471 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701474 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701476 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701479 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701482 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701484 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701487 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701490 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701492 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701495 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701498 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701500 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701504 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701506 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701526 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701529 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701532 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701534 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701537 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701540 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:57:49.831825 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701542 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701545 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701548 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701550 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701553 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701556 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701558 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701561 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701563 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701566 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701568 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701571 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701573 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701576 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701579 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701581 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701584 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701586 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701589 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701591 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:57:49.832477 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701594 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701597 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701599 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701602 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701606 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701609 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701612 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701615 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701617 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701620 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701622 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701625 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701628 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701630 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701634 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701637 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701640 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701643 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701646 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:57:49.833119 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701648 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701651 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701653 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701656 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701658 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701661 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701665 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701668 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701671 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701674 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701677 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701679 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701682 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701685 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701687 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701689 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701693 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701697 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:57:49.833775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.701699 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:57:49.834401 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.702203 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 02:57:49.834401 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.712005 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 02:57:49.834401 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.712020 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 02:57:49.834401 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.712038 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 02:57:49.834401 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.712044 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 02:57:49.834401 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712101 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:57:49.834401 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712107 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:57:49.834401 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712110 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:57:49.834401 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712113 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:57:49.834401 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712116 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:57:49.834401 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712119 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:57:49.834401 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712121 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:57:49.834401 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712124 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:57:49.834401 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712127 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:57:49.834401 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712129 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712132 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712135 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712138 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712140 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712143 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712145 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712148 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712151 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712153 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712156 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712158 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712160 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712165 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712169 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712174 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712177 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712181 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712185 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:57:49.835027 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712188 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712191 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712194 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712196 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712199 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712201 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712204 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712206 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712211 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712215 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712219 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712222 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712225 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712228 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712230 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712233 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712235 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712238 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712241 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:57:49.835858 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712243 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712246 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712249 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712251 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712254 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712258 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712261 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712265 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712268 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712270 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712273 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712276 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712279 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712281 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712284 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712286 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712289 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712291 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712294 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712296 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:57:49.836423 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712299 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712301 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712304 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712306 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712310 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712314 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712317 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712319 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712322 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712324 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712327 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712329 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712332 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712334 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712337 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712339 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712341 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712345 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712348 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:57:49.837043 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712352 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:57:49.837675 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712354 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:57:49.837675 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712357 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:57:49.837675 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712359 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:57:49.837675 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712362 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:57:49.837675 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712365 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:57:49.837675 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712367 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:57:49.837675 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712369 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:57:49.837675 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712372 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:57:49.837675 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712375 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:57:49.837675 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712377 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:57:49.837675 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712380 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:57:49.837675 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712382 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:57:49.837675 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.712387 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 02:57:49.837675 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712509 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:57:49.837675 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712529 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712532 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712535 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712538 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712541 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712543 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712546 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712549 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712552 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712554 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712557 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712560 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712562 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712565 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712568 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712572 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712575 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712577 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712582 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712584 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:57:49.838077 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712587 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712589 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712593 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712596 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712598 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712601 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712603 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712606 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712609 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712613 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712617 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712620 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712623 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712626 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712629 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712631 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712634 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712636 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712639 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712641 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:57:49.838772 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712644 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712646 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712649 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712651 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712653 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712656 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712660 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712665 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712668 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712670 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712674 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712677 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712680 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712683 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712686 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712689 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712692 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712695 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712697 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712700 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:57:49.839429 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712702 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712705 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712707 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712710 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712712 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712715 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712717 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712720 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712722 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712725 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712727 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712730 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712732 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712735 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712737 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712740 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712742 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712745 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712747 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:57:49.840128 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712751 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712753 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712756 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712760 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712764 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712767 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712770 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712773 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712776 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712779 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712781 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712784 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712786 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712789 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712791 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712794 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712796 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712799 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:57:49.840655 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:49.712801 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:57:49.841163 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.712806 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 02:57:49.841163 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.712903 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 02:57:49.841163 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.714911 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 02:57:49.841163 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.716019 2580 server.go:1019] "Starting client certificate rotation" Jul 22 02:57:49.841163 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.716139 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 02:57:49.841163 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.716166 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 02:57:49.841163 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.718103 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 02:57:49.841163 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.721912 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 02:57:49.841163 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.738540 2580 log.go:25] "Validated CRI v1 runtime API" Jul 22 02:57:49.841163 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.741189 2580 log.go:25] "Validated CRI v1 image API" Jul 22 02:57:49.841163 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.741979 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 02:57:49.841163 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.742998 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 02:57:49.841640 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.746579 2580 fs.go:135] Filesystem UUIDs: map[5c57f0b6-5848-44c7-9201-2fa351a71f07:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f8174d43-9312-4190-99f5-093a16f546ab:/dev/nvme0n1p4] Jul 22 02:57:49.841640 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.746598 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.752048 2580 manager.go:216] Machine: {Timestamp:2026-07-22 02:57:49.750899975 +0000 UTC m=+0.340410667 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3117806 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f50d8ac490a714c5f425c534cc6ca SystemUUID:ec2f50d8-ac49-0a71-4c5f-425c534cc6ca BootID:9b2bfca6-bcb8-46ca-aed1-840cb197ceca Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ed:67:c3:e8:fd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ed:67:c3:e8:fd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:af:82:0a:b4:aa Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.752142 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.752240 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.753744 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.753766 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-55.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.754813 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.754823 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.754830 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.755358 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.756588 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.760587 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.760705 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.760733 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.760744 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.760755 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.760763 2580 kubelet.go:398] "Adding apiserver pod source" Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.760772 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 02:57:49.946397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.761082 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.761097 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.763298 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.765155 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.765182 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.766260 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.766276 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.766282 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.766288 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.766297 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.766303 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.766310 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.766316 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.766323 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.766328 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.766337 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.766342 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.766929 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.766939 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.770286 2580 server.go:1268] "Started kubelet" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:49.770968 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-55.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:49.771044 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.771106 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-55.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.771154 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.781454 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.781503 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.782844 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.783173 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.796826 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8jplh" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:49.798951 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:49.801208 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-55.ec2.internal.18c47d40d62dbeb9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-55.ec2.internal,UID:ip-10-0-138-55.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-55.ec2.internal,},FirstTimestamp:2026-07-22 02:57:49.770260153 +0000 UTC m=+0.359770845,LastTimestamp:2026-07-22 02:57:49.770260153 +0000 UTC m=+0.359770845,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-55.ec2.internal,}" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.804571 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.804582 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:49.804653 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-55.ec2.internal\" not found" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.805049 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.805053 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.805078 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.805203 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.805215 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.805613 2580 factory.go:153] Registering CRI-O factory Jul 22 02:57:49.970488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.805629 2580 factory.go:223] Registration of the crio container factory successfully Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.805689 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.805698 2580 factory.go:55] Registering systemd factory Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.805705 2580 factory.go:223] Registration of the systemd container factory successfully Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.805724 2580 factory.go:103] Registering Raw factory Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.805737 2580 manager.go:1196] Started watching for new ooms in manager Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.806150 2580 manager.go:349] Starting recovery of all containers Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.806504 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8jplh" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.816471 2580 manager.go:354] Recovery completed Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.817561 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:49.819909 2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-55.ec2.internal\" not found" node="ip-10-0-138-55.ec2.internal" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.820705 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.823089 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.823114 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.823125 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.823696 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.823705 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.823704 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.823773 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.824420 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.824448 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.824459 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.825821 2580 policy_none.go:49] "None policy: Start" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.825836 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.825848 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.827842 2580 policy_none.go:47] "Start" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.842597 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.874077 2580 manager.go:337] "Starting Device Plugin manager" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:49.874103 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.874112 2580 server.go:87] "Starting device plugin registration server" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.874338 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.874368 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.874482 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.874586 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.874594 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:49.875023 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:49.875073 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-55.ec2.internal\" not found" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.962039 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.962201 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.962226 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:49.962257 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.964709 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:57:49.977645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.977637 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:57:49.978932 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.978423 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:57:49.978932 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.978452 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:57:49.978932 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.978466 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:57:49.978932 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.978495 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-55.ec2.internal" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:49.987567 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-55.ec2.internal" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:49.987585 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-55.ec2.internal\": node \"ip-10-0-138-55.ec2.internal\" not found" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:50.009959 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-55.ec2.internal\" not found" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.062545 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-55.ec2.internal"] Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.062620 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.063503 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.063539 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.063548 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.065936 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.066064 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.066093 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.066598 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.066606 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.066620 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.066630 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.066643 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.066630 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.068935 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-55.ec2.internal" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.068955 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.069618 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.069636 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:50.069646 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:50.085030 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-55.ec2.internal\" not found" node="ip-10-0-138-55.ec2.internal" Jul 22 02:57:50.092033 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:50.088949 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-55.ec2.internal\" not found" node="ip-10-0-138-55.ec2.internal" Jul 22 02:57:51.615158 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:51.504254 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1bb2de6367befd29ddcad04f7dc49a28-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal\" (UID: \"1bb2de6367befd29ddcad04f7dc49a28\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal" Jul 22 02:57:51.615158 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:51.504274 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1bb2de6367befd29ddcad04f7dc49a28-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal\" (UID: \"1bb2de6367befd29ddcad04f7dc49a28\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal" Jul 22 02:57:51.615158 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:51.504291 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/671074e8de9b7c15ed7884f108ed8b26-config\") pod \"kube-apiserver-proxy-ip-10-0-138-55.ec2.internal\" (UID: \"671074e8de9b7c15ed7884f108ed8b26\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-55.ec2.internal" Jul 22 02:57:51.615158 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:51.509293 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-55.ec2.internal\" not found" Jul 22 02:57:51.615158 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:51.604465 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1bb2de6367befd29ddcad04f7dc49a28-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal\" (UID: \"1bb2de6367befd29ddcad04f7dc49a28\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal" Jul 22 02:57:51.615158 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:51.604494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1bb2de6367befd29ddcad04f7dc49a28-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal\" (UID: \"1bb2de6367befd29ddcad04f7dc49a28\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal" Jul 22 02:57:51.615158 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:51.604515 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/671074e8de9b7c15ed7884f108ed8b26-config\") pod \"kube-apiserver-proxy-ip-10-0-138-55.ec2.internal\" (UID: \"671074e8de9b7c15ed7884f108ed8b26\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-55.ec2.internal" Jul 22 02:57:51.615158 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:51.604567 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1bb2de6367befd29ddcad04f7dc49a28-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal\" (UID: \"1bb2de6367befd29ddcad04f7dc49a28\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal" Jul 22 02:57:51.615158 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:51.604586 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1bb2de6367befd29ddcad04f7dc49a28-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal\" (UID: \"1bb2de6367befd29ddcad04f7dc49a28\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal" Jul 22 02:57:51.615158 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:51.604567 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/671074e8de9b7c15ed7884f108ed8b26-config\") pod \"kube-apiserver-proxy-ip-10-0-138-55.ec2.internal\" (UID: \"671074e8de9b7c15ed7884f108ed8b26\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-55.ec2.internal" Jul 22 02:57:51.615158 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:51.609580 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-55.ec2.internal\" not found" Jul 22 02:57:51.741021 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:51.710216 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-55.ec2.internal\" not found" Jul 22 02:57:51.850402 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:51.786428 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal" Jul 22 02:57:51.850402 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:51.790099 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-55.ec2.internal" Jul 22 02:57:51.850402 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:51.811271 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-55.ec2.internal\" not found" Jul 22 02:57:51.970426 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:51.912124 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-55.ec2.internal\" not found" Jul 22 02:57:52.081619 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:52.012649 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-55.ec2.internal\" not found" Jul 22 02:57:52.188069 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:52.113088 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-55.ec2.internal\" not found" Jul 22 02:57:52.188069 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:52.114169 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 02:57:52.188069 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:52.114285 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 02:57:52.188069 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:52.114322 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 02:57:52.314090 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:52.198990 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:57:52.314090 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:52.203859 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal" Jul 22 02:57:52.314090 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:52.203879 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 02:57:52.314090 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:52.209736 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 02:52:51 +0000 UTC" deadline="2028-03-12 16:38:56.97239031 +0000 UTC" Jul 22 02:57:52.314090 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:52.209754 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14389h41m4.762638794s" Jul 22 02:57:52.314090 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:52.213959 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 02:57:52.314090 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:52.218096 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 02:57:52.314090 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:52.219594 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-55.ec2.internal" Jul 22 02:57:52.314090 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:52.226384 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 02:57:52.314090 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:52.236767 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xvfvh" Jul 22 02:57:52.314090 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:52.244918 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xvfvh" Jul 22 02:57:52.465678 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:52.423888 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:57:52.665904 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:52.560248 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bb2de6367befd29ddcad04f7dc49a28.slice/crio-ec9c85e81bd962200fd542303b42cd98e557feffbb0708c5a889659f3525e889 WatchSource:0}: Error finding container ec9c85e81bd962200fd542303b42cd98e557feffbb0708c5a889659f3525e889: Status 404 returned error can't find the container with id ec9c85e81bd962200fd542303b42cd98e557feffbb0708c5a889659f3525e889 Jul 22 02:57:52.665904 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:52.560782 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod671074e8de9b7c15ed7884f108ed8b26.slice/crio-c09d34657d761510c92eb1f4e3879db9cc3219b56671bc66723b13017eeda6d2 WatchSource:0}: Error finding container c09d34657d761510c92eb1f4e3879db9cc3219b56671bc66723b13017eeda6d2: Status 404 returned error can't find the container with id c09d34657d761510c92eb1f4e3879db9cc3219b56671bc66723b13017eeda6d2 Jul 22 02:57:52.665904 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:52.564416 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:57:52.973129 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:52.973042 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:57:53.160488 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.160459 2580 apiserver.go:52] "Watching apiserver" Jul 22 02:57:53.166649 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.166623 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 02:57:53.167282 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.167261 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-t8ln6","kube-system/konnectivity-agent-qk5x9","openshift-dns/node-resolver-7t98c","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal","openshift-multus/network-metrics-daemon-mlxzc","openshift-network-diagnostics/network-check-target-szjw4","openshift-ovn-kubernetes/ovnkube-node-zptvl","kube-system/global-pull-secret-syncer-9tw9c","kube-system/kube-apiserver-proxy-ip-10-0-138-55.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj","openshift-cluster-node-tuning-operator/tuned-h2k5h","openshift-image-registry/node-ca-99vc7","openshift-multus/multus-8mtj2","openshift-multus/multus-additional-cni-plugins-ch9w2"] Jul 22 02:57:53.170302 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.170279 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.173064 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.173018 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.173518 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.173484 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 02:57:53.173518 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.173504 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 02:57:53.173658 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.173538 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 02:57:53.173824 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.173794 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5kn59\"" Jul 22 02:57:53.175289 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.175041 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 02:57:53.175289 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.175060 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 02:57:53.175429 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.175291 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 02:57:53.175429 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.175305 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 02:57:53.175429 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.175350 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nhb88\"" Jul 22 02:57:53.175773 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.175754 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7t98c" Jul 22 02:57:53.177667 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.177647 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 02:57:53.177814 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.177793 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ct7fj\"" Jul 22 02:57:53.177962 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.177930 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.178153 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.178132 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 02:57:53.180360 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.180290 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 02:57:53.180360 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.180355 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 02:57:53.180551 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.180528 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:57:53.180618 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.180587 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vzz55\"" Jul 22 02:57:53.180618 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.180597 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:57:53.182962 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.182399 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:57:53.182962 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.182456 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:57:53.184696 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.184677 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:57:53.185021 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.184996 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:57:53.187201 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.187180 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.189628 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.189609 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qk5x9" Jul 22 02:57:53.191501 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.191328 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 02:57:53.191637 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.191623 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5ch4h\"" Jul 22 02:57:53.191728 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.191711 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 02:57:53.192278 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.192042 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t8ln6" Jul 22 02:57:53.193503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.192533 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 02:57:53.193503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.192611 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 02:57:53.193503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.192875 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 02:57:53.193503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.193108 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vcmq5\"" Jul 22 02:57:53.193503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.193311 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 02:57:53.193503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.193340 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 02:57:53.193864 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.193583 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 02:57:53.194423 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.194330 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mkqr4\"" Jul 22 02:57:53.194526 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.194435 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 02:57:53.194601 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.194549 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:57:53.194799 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.194752 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 02:57:53.195063 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.194975 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.197234 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.197096 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-99vc7" Jul 22 02:57:53.198455 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.198437 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 02:57:53.198564 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.198548 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nlh5f\"" Jul 22 02:57:53.198652 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.198636 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:57:53.200719 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.200388 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 02:57:53.200719 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.200496 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 02:57:53.200719 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.200575 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 02:57:53.200719 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.200586 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zkbfs\"" Jul 22 02:57:53.206623 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.206486 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 02:57:53.214296 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.214273 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-multus-socket-dir-parent\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.214645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.214625 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/308ab8f1-8fb2-4833-b583-b22cf8e3cddd-tmp-dir\") pod \"node-resolver-7t98c\" (UID: \"308ab8f1-8fb2-4833-b583-b22cf8e3cddd\") " pod="openshift-dns/node-resolver-7t98c" Jul 22 02:57:53.214941 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.214923 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-run-netns\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.215158 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.215141 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-log-socket\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.215317 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.215301 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-run-ovn-kubernetes\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.215759 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.215741 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhxql\" (UniqueName: \"kubernetes.io/projected/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-kube-api-access-bhxql\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.216048 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216028 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bkkh\" (UniqueName: \"kubernetes.io/projected/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-kube-api-access-5bkkh\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.216245 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216181 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2nkg\" (UniqueName: \"kubernetes.io/projected/faa320f4-3fb1-4b07-b021-2f00bcff4f88-kube-api-access-v2nkg\") pod \"iptables-alerter-t8ln6\" (UID: \"faa320f4-3fb1-4b07-b021-2f00bcff4f88\") " pod="openshift-network-operator/iptables-alerter-t8ln6" Jul 22 02:57:53.216418 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216400 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-node-log\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.216529 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216429 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e9a37af0-c577-48af-9af1-c6089fb43ac2-env-overrides\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.216529 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216453 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-tuned\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.216529 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216475 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-etc-selinux\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.216529 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216498 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-multus-conf-dir\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.216529 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216523 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-run-multus-certs\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.216792 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216545 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxhc2\" (UniqueName: \"kubernetes.io/projected/d3473130-eefa-4dfb-82b6-7b464489959f-kube-api-access-wxhc2\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.216792 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216570 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/183fb194-f313-4528-bf1b-c457f3174d9d-kubelet-config\") pod \"global-pull-secret-syncer-9tw9c\" (UID: \"183fb194-f313-4528-bf1b-c457f3174d9d\") " pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:57:53.216792 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216594 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-run-ovn\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.216792 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216642 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.216792 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216681 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-system-cni-dir\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.216792 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.216792 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216752 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zlmt\" (UniqueName: \"kubernetes.io/projected/e9a37af0-c577-48af-9af1-c6089fb43ac2-kube-api-access-5zlmt\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.217070 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216818 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-sysctl-conf\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.217070 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216874 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-var-lib-cni-multus\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.217070 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216904 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.217070 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216936 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs\") pod \"network-metrics-daemon-mlxzc\" (UID: \"31d06428-0968-400f-8b93-86bacf6ec123\") " pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:57:53.217070 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216960 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/faa320f4-3fb1-4b07-b021-2f00bcff4f88-host-slash\") pod \"iptables-alerter-t8ln6\" (UID: \"faa320f4-3fb1-4b07-b021-2f00bcff4f88\") " pod="openshift-network-operator/iptables-alerter-t8ln6" Jul 22 02:57:53.217070 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.216983 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-systemd-units\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.217070 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217013 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e9a37af0-c577-48af-9af1-c6089fb43ac2-ovnkube-script-lib\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.217597 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217045 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-sysconfig\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.217597 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217104 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-multus-cni-dir\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.217597 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217127 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/faa320f4-3fb1-4b07-b021-2f00bcff4f88-iptables-alerter-script\") pod \"iptables-alerter-t8ln6\" (UID: \"faa320f4-3fb1-4b07-b021-2f00bcff4f88\") " pod="openshift-network-operator/iptables-alerter-t8ln6" Jul 22 02:57:53.217597 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217173 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-slash\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.217597 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217195 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-etc-openvswitch\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.217597 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217218 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-sysctl-d\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.217597 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217254 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-var-lib-cni-bin\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.217597 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217289 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-etc-kubernetes\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.217597 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217310 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/308ab8f1-8fb2-4833-b583-b22cf8e3cddd-hosts-file\") pod \"node-resolver-7t98c\" (UID: \"308ab8f1-8fb2-4833-b583-b22cf8e3cddd\") " pod="openshift-dns/node-resolver-7t98c" Jul 22 02:57:53.217597 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217331 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-cni-bin\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.217597 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217354 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e9a37af0-c577-48af-9af1-c6089fb43ac2-ovn-node-metrics-cert\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.217597 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217384 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6z2z\" (UniqueName: \"kubernetes.io/projected/91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b-kube-api-access-f6z2z\") pod \"node-ca-99vc7\" (UID: \"91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b\") " pod="openshift-image-registry/node-ca-99vc7" Jul 22 02:57:53.217597 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217413 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tw9c\" (UID: \"183fb194-f313-4528-bf1b-c457f3174d9d\") " pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:57:53.217597 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217436 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e9a37af0-c577-48af-9af1-c6089fb43ac2-ovnkube-config\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.217597 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217467 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75e0da4f-f558-49ae-969f-14dcb906ebe6-tmp\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.217597 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217502 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b-serviceca\") pod \"node-ca-99vc7\" (UID: \"91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b\") " pod="openshift-image-registry/node-ca-99vc7" Jul 22 02:57:53.217597 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217524 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-device-dir\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.218336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217548 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3473130-eefa-4dfb-82b6-7b464489959f-cni-binary-copy\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.218336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217571 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-run-k8s-cni-cncf-io\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.218336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217595 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-kubelet\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.218336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217617 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b-host\") pod \"node-ca-99vc7\" (UID: \"91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b\") " pod="openshift-image-registry/node-ca-99vc7" Jul 22 02:57:53.218336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217640 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-registration-dir\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.218336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217662 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-run-netns\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.218336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217684 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-var-lib-kubelet\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.218336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217706 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d3473130-eefa-4dfb-82b6-7b464489959f-multus-daemon-config\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.218336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217729 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-cni-binary-copy\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.218336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217752 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.218336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217792 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-cnibin\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.218336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217822 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-cni-netd\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.218336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217861 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-modprobe-d\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.218336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217884 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-run\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.218336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217904 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-os-release\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.218336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217925 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-hostroot\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.218336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.217963 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-systemd\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.219139 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218007 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-sys\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.219139 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218036 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4frm\" (UniqueName: \"kubernetes.io/projected/31d06428-0968-400f-8b93-86bacf6ec123-kube-api-access-z4frm\") pod \"network-metrics-daemon-mlxzc\" (UID: \"31d06428-0968-400f-8b93-86bacf6ec123\") " pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:57:53.219139 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218152 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/183fb194-f313-4528-bf1b-c457f3174d9d-dbus\") pod \"global-pull-secret-syncer-9tw9c\" (UID: \"183fb194-f313-4528-bf1b-c457f3174d9d\") " pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:57:53.219139 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218182 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-run-systemd\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.219139 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218204 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-run-openvswitch\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.219139 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218226 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0625b87a-9139-4d7e-9a02-78c535b9473a-agent-certs\") pod \"konnectivity-agent-qk5x9\" (UID: \"0625b87a-9139-4d7e-9a02-78c535b9473a\") " pod="kube-system/konnectivity-agent-qk5x9" Jul 22 02:57:53.219139 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218265 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-kubernetes\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.219139 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218296 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-lib-modules\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.219139 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218320 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-system-cni-dir\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.219139 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218349 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-cnibin\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.219139 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218372 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-var-lib-openvswitch\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.219139 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218393 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-socket-dir\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.219139 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218416 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-os-release\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.219139 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218447 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.219139 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218475 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xwcm\" (UniqueName: \"kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm\") pod \"network-check-target-szjw4\" (UID: \"5588cb65-fb3d-4955-98d4-5f5d5ad5234c\") " pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:57:53.219139 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218507 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-var-lib-kubelet\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.219945 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218529 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-sys-fs\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.219945 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218556 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk6rr\" (UniqueName: \"kubernetes.io/projected/308ab8f1-8fb2-4833-b583-b22cf8e3cddd-kube-api-access-dk6rr\") pod \"node-resolver-7t98c\" (UID: \"308ab8f1-8fb2-4833-b583-b22cf8e3cddd\") " pod="openshift-dns/node-resolver-7t98c" Jul 22 02:57:53.219945 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218579 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0625b87a-9139-4d7e-9a02-78c535b9473a-konnectivity-ca\") pod \"konnectivity-agent-qk5x9\" (UID: \"0625b87a-9139-4d7e-9a02-78c535b9473a\") " pod="kube-system/konnectivity-agent-qk5x9" Jul 22 02:57:53.219945 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218603 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-host\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.219945 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.218658 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxxzj\" (UniqueName: \"kubernetes.io/projected/75e0da4f-f558-49ae-969f-14dcb906ebe6-kube-api-access-vxxzj\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.247635 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.247583 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 02:52:52 +0000 UTC" deadline="2028-04-12 23:03:26.604266538 +0000 UTC" Jul 22 02:57:53.247635 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.247634 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15140h5m33.356636742s" Jul 22 02:57:53.302543 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.302513 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:57:53.319168 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319136 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/308ab8f1-8fb2-4833-b583-b22cf8e3cddd-tmp-dir\") pod \"node-resolver-7t98c\" (UID: \"308ab8f1-8fb2-4833-b583-b22cf8e3cddd\") " pod="openshift-dns/node-resolver-7t98c" Jul 22 02:57:53.319341 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319180 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-run-netns\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.319341 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319199 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-log-socket\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.319341 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319215 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-run-ovn-kubernetes\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.319341 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319234 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhxql\" (UniqueName: \"kubernetes.io/projected/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-kube-api-access-bhxql\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.319341 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319259 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bkkh\" (UniqueName: \"kubernetes.io/projected/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-kube-api-access-5bkkh\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.319341 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319285 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2nkg\" (UniqueName: \"kubernetes.io/projected/faa320f4-3fb1-4b07-b021-2f00bcff4f88-kube-api-access-v2nkg\") pod \"iptables-alerter-t8ln6\" (UID: \"faa320f4-3fb1-4b07-b021-2f00bcff4f88\") " pod="openshift-network-operator/iptables-alerter-t8ln6" Jul 22 02:57:53.319341 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319342 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-node-log\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.319671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319359 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e9a37af0-c577-48af-9af1-c6089fb43ac2-env-overrides\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.319671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319373 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-tuned\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.319671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319389 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-etc-selinux\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.319671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319411 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-multus-conf-dir\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.319671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319427 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-run-multus-certs\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.319671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319453 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxhc2\" (UniqueName: \"kubernetes.io/projected/d3473130-eefa-4dfb-82b6-7b464489959f-kube-api-access-wxhc2\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.319671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319468 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/183fb194-f313-4528-bf1b-c457f3174d9d-kubelet-config\") pod \"global-pull-secret-syncer-9tw9c\" (UID: \"183fb194-f313-4528-bf1b-c457f3174d9d\") " pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:57:53.319671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319490 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-run-ovn\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.319671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319511 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.319671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319523 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/308ab8f1-8fb2-4833-b583-b22cf8e3cddd-tmp-dir\") pod \"node-resolver-7t98c\" (UID: \"308ab8f1-8fb2-4833-b583-b22cf8e3cddd\") " pod="openshift-dns/node-resolver-7t98c" Jul 22 02:57:53.319671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319581 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-log-socket\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.319671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319622 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-run-ovn-kubernetes\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.319671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319626 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-system-cni-dir\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.319671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319530 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-system-cni-dir\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.320397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319675 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.320397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zlmt\" (UniqueName: \"kubernetes.io/projected/e9a37af0-c577-48af-9af1-c6089fb43ac2-kube-api-access-5zlmt\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.320397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319729 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-sysctl-conf\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.320397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319755 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-var-lib-cni-multus\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.320397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319781 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.320397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319809 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs\") pod \"network-metrics-daemon-mlxzc\" (UID: \"31d06428-0968-400f-8b93-86bacf6ec123\") " pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:57:53.320397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319852 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/faa320f4-3fb1-4b07-b021-2f00bcff4f88-host-slash\") pod \"iptables-alerter-t8ln6\" (UID: \"faa320f4-3fb1-4b07-b021-2f00bcff4f88\") " pod="openshift-network-operator/iptables-alerter-t8ln6" Jul 22 02:57:53.320397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319877 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-systemd-units\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.320397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319894 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-node-log\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.320397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.319900 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e9a37af0-c577-48af-9af1-c6089fb43ac2-ovnkube-script-lib\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.320397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320268 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-etc-selinux\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.320397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320314 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-run-ovn\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.320397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320321 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-sysconfig\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.320397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320336 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e9a37af0-c577-48af-9af1-c6089fb43ac2-env-overrides\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.320397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320368 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-multus-cni-dir\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.320397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320401 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/faa320f4-3fb1-4b07-b021-2f00bcff4f88-iptables-alerter-script\") pod \"iptables-alerter-t8ln6\" (UID: \"faa320f4-3fb1-4b07-b021-2f00bcff4f88\") " pod="openshift-network-operator/iptables-alerter-t8ln6" Jul 22 02:57:53.320397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320408 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320427 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-slash\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320443 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-var-lib-cni-multus\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320452 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-etc-openvswitch\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320468 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320475 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-sysctl-d\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320495 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-sysctl-conf\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320569 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-var-lib-cni-bin\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320601 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-etc-kubernetes\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320605 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-sysctl-d\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320629 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/308ab8f1-8fb2-4833-b583-b22cf8e3cddd-hosts-file\") pod \"node-resolver-7t98c\" (UID: \"308ab8f1-8fb2-4833-b583-b22cf8e3cddd\") " pod="openshift-dns/node-resolver-7t98c" Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320652 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-var-lib-cni-bin\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320655 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-cni-bin\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320367 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-run-multus-certs\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320679 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320681 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e9a37af0-c577-48af-9af1-c6089fb43ac2-ovn-node-metrics-cert\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320720 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6z2z\" (UniqueName: \"kubernetes.io/projected/91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b-kube-api-access-f6z2z\") pod \"node-ca-99vc7\" (UID: \"91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b\") " pod="openshift-image-registry/node-ca-99vc7" Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.320744 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:57:53.321263 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.320800 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:57:53.322191 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.320850 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs podName:31d06428-0968-400f-8b93-86bacf6ec123 nodeName:}" failed. No retries permitted until 2026-07-22 02:57:53.820794 +0000 UTC m=+3.011988297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs") pod "network-metrics-daemon-mlxzc" (UID: "31d06428-0968-400f-8b93-86bacf6ec123") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:57:53.322191 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.320869 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret podName:183fb194-f313-4528-bf1b-c457f3174d9d nodeName:}" failed. No retries permitted until 2026-07-22 02:57:53.820857589 +0000 UTC m=+3.012051873 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret") pod "global-pull-secret-syncer-9tw9c" (UID: "183fb194-f313-4528-bf1b-c457f3174d9d") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:57:53.322191 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320901 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/faa320f4-3fb1-4b07-b021-2f00bcff4f88-host-slash\") pod \"iptables-alerter-t8ln6\" (UID: \"faa320f4-3fb1-4b07-b021-2f00bcff4f88\") " pod="openshift-network-operator/iptables-alerter-t8ln6" Jul 22 02:57:53.322191 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320941 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-systemd-units\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.322191 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320986 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-sysconfig\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.322191 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321018 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.322191 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321087 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/308ab8f1-8fb2-4833-b583-b22cf8e3cddd-hosts-file\") pod \"node-resolver-7t98c\" (UID: \"308ab8f1-8fb2-4833-b583-b22cf8e3cddd\") " pod="openshift-dns/node-resolver-7t98c" Jul 22 02:57:53.322191 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320745 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tw9c\" (UID: \"183fb194-f313-4528-bf1b-c457f3174d9d\") " pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:57:53.322191 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321143 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-etc-kubernetes\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.322191 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321149 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e9a37af0-c577-48af-9af1-c6089fb43ac2-ovnkube-config\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.322191 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321162 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-multus-cni-dir\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.322191 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321180 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75e0da4f-f558-49ae-969f-14dcb906ebe6-tmp\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.322191 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321194 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-cni-bin\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.322191 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320359 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/183fb194-f313-4528-bf1b-c457f3174d9d-kubelet-config\") pod \"global-pull-secret-syncer-9tw9c\" (UID: \"183fb194-f313-4528-bf1b-c457f3174d9d\") " pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:57:53.322191 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321213 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-multus-conf-dir\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.322191 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321207 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b-serviceca\") pod \"node-ca-99vc7\" (UID: \"91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b\") " pod="openshift-image-registry/node-ca-99vc7" Jul 22 02:57:53.322920 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321255 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-device-dir\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.322920 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321323 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3473130-eefa-4dfb-82b6-7b464489959f-cni-binary-copy\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.322920 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321354 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-run-k8s-cni-cncf-io\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.322920 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321364 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-device-dir\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.322920 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321379 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-kubelet\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.322920 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321412 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b-host\") pod \"node-ca-99vc7\" (UID: \"91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b\") " pod="openshift-image-registry/node-ca-99vc7" Jul 22 02:57:53.322920 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321417 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-kubelet\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.322920 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321426 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-slash\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.322920 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321447 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-registration-dir\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.322920 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321451 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b-host\") pod \"node-ca-99vc7\" (UID: \"91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b\") " pod="openshift-image-registry/node-ca-99vc7" Jul 22 02:57:53.322920 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321472 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-run-netns\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.322920 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321495 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-var-lib-kubelet\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.322920 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321507 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/faa320f4-3fb1-4b07-b021-2f00bcff4f88-iptables-alerter-script\") pod \"iptables-alerter-t8ln6\" (UID: \"faa320f4-3fb1-4b07-b021-2f00bcff4f88\") " pod="openshift-network-operator/iptables-alerter-t8ln6" Jul 22 02:57:53.322920 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321520 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d3473130-eefa-4dfb-82b6-7b464489959f-multus-daemon-config\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.322920 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321544 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-cni-binary-copy\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.322920 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321567 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.322920 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321569 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-etc-openvswitch\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321594 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-cnibin\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321617 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-cni-netd\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321621 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-var-lib-kubelet\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321647 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b-serviceca\") pod \"node-ca-99vc7\" (UID: \"91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b\") " pod="openshift-image-registry/node-ca-99vc7" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321708 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-cnibin\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321729 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-modprobe-d\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321760 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-registration-dir\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321784 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321652 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-modprobe-d\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321804 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-run-k8s-cni-cncf-io\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321846 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-run\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321874 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-os-release\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321898 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-hostroot\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321920 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-systemd\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321942 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-sys\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321962 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d3473130-eefa-4dfb-82b6-7b464489959f-cni-binary-copy\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322035 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-os-release\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.323698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322084 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-hostroot\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322126 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-systemd\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.321966 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4frm\" (UniqueName: \"kubernetes.io/projected/31d06428-0968-400f-8b93-86bacf6ec123-kube-api-access-z4frm\") pod \"network-metrics-daemon-mlxzc\" (UID: \"31d06428-0968-400f-8b93-86bacf6ec123\") " pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322169 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-sys\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322182 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/d3473130-eefa-4dfb-82b6-7b464489959f-multus-daemon-config\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322195 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/183fb194-f313-4528-bf1b-c457f3174d9d-dbus\") pod \"global-pull-secret-syncer-9tw9c\" (UID: \"183fb194-f313-4528-bf1b-c457f3174d9d\") " pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322207 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e9a37af0-c577-48af-9af1-c6089fb43ac2-ovnkube-script-lib\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322233 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-run-systemd\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322248 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/183fb194-f313-4528-bf1b-c457f3174d9d-dbus\") pod \"global-pull-secret-syncer-9tw9c\" (UID: \"183fb194-f313-4528-bf1b-c457f3174d9d\") " pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322263 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-run-openvswitch\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322266 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-host-run-netns\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322289 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-cni-netd\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322305 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-run-systemd\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322289 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0625b87a-9139-4d7e-9a02-78c535b9473a-agent-certs\") pod \"konnectivity-agent-qk5x9\" (UID: \"0625b87a-9139-4d7e-9a02-78c535b9473a\") " pod="kube-system/konnectivity-agent-qk5x9" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322333 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-run-openvswitch\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322341 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-kubernetes\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322369 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-lib-modules\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322387 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-kubernetes\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.324671 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322391 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-system-cni-dir\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.325503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322420 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-cnibin\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.325503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322431 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-system-cni-dir\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.325503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322447 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-var-lib-openvswitch\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.325503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322475 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-socket-dir\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.325503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322498 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-os-release\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.325503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322525 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.325503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322536 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-lib-modules\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.325503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322551 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xwcm\" (UniqueName: \"kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm\") pod \"network-check-target-szjw4\" (UID: \"5588cb65-fb3d-4955-98d4-5f5d5ad5234c\") " pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:57:53.325503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322577 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-var-lib-kubelet\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.325503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322600 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-sys-fs\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.325503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322626 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk6rr\" (UniqueName: \"kubernetes.io/projected/308ab8f1-8fb2-4833-b583-b22cf8e3cddd-kube-api-access-dk6rr\") pod \"node-resolver-7t98c\" (UID: \"308ab8f1-8fb2-4833-b583-b22cf8e3cddd\") " pod="openshift-dns/node-resolver-7t98c" Jul 22 02:57:53.325503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322649 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0625b87a-9139-4d7e-9a02-78c535b9473a-konnectivity-ca\") pod \"konnectivity-agent-qk5x9\" (UID: \"0625b87a-9139-4d7e-9a02-78c535b9473a\") " pod="kube-system/konnectivity-agent-qk5x9" Jul 22 02:57:53.325503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322671 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-socket-dir\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.325503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322677 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-cni-binary-copy\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.325503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322675 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-host\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.325503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322712 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-cnibin\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.325503 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322719 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-host\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.326441 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322731 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxxzj\" (UniqueName: \"kubernetes.io/projected/75e0da4f-f558-49ae-969f-14dcb906ebe6-kube-api-access-vxxzj\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.326441 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322749 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-var-lib-openvswitch\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.326441 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322758 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-multus-socket-dir-parent\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.326441 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322774 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-os-release\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.326441 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322854 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/d3473130-eefa-4dfb-82b6-7b464489959f-multus-socket-dir-parent\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.326441 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322900 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-run\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.326441 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.322930 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-sys-fs\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.326441 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.323067 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e9a37af0-c577-48af-9af1-c6089fb43ac2-ovnkube-config\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.326441 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.323101 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/75e0da4f-f558-49ae-969f-14dcb906ebe6-var-lib-kubelet\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.326441 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.320475 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e9a37af0-c577-48af-9af1-c6089fb43ac2-host-run-netns\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.326441 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.323394 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.326441 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.323554 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0625b87a-9139-4d7e-9a02-78c535b9473a-konnectivity-ca\") pod \"konnectivity-agent-qk5x9\" (UID: \"0625b87a-9139-4d7e-9a02-78c535b9473a\") " pod="kube-system/konnectivity-agent-qk5x9" Jul 22 02:57:53.326441 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.325648 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75e0da4f-f558-49ae-969f-14dcb906ebe6-tmp\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.326441 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.325889 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0625b87a-9139-4d7e-9a02-78c535b9473a-agent-certs\") pod \"konnectivity-agent-qk5x9\" (UID: \"0625b87a-9139-4d7e-9a02-78c535b9473a\") " pod="kube-system/konnectivity-agent-qk5x9" Jul 22 02:57:53.327145 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.327123 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/75e0da4f-f558-49ae-969f-14dcb906ebe6-etc-tuned\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.328329 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.328067 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e9a37af0-c577-48af-9af1-c6089fb43ac2-ovn-node-metrics-cert\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.329233 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.329208 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bkkh\" (UniqueName: \"kubernetes.io/projected/0ae2fd4b-4b95-4e25-832f-8e2599d50b8d-kube-api-access-5bkkh\") pod \"multus-additional-cni-plugins-ch9w2\" (UID: \"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d\") " pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.330285 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.330261 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhxql\" (UniqueName: \"kubernetes.io/projected/e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539-kube-api-access-bhxql\") pod \"aws-ebs-csi-driver-node-d2bzj\" (UID: \"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.332210 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.332057 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2nkg\" (UniqueName: \"kubernetes.io/projected/faa320f4-3fb1-4b07-b021-2f00bcff4f88-kube-api-access-v2nkg\") pod \"iptables-alerter-t8ln6\" (UID: \"faa320f4-3fb1-4b07-b021-2f00bcff4f88\") " pod="openshift-network-operator/iptables-alerter-t8ln6" Jul 22 02:57:53.336557 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.334853 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:57:53.336557 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.334881 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:57:53.336557 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.334896 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5xwcm for pod openshift-network-diagnostics/network-check-target-szjw4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:57:53.336557 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.334964 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm podName:5588cb65-fb3d-4955-98d4-5f5d5ad5234c nodeName:}" failed. No retries permitted until 2026-07-22 02:57:53.834942124 +0000 UTC m=+3.026136406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5xwcm" (UniqueName: "kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm") pod "network-check-target-szjw4" (UID: "5588cb65-fb3d-4955-98d4-5f5d5ad5234c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:57:53.339267 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.339196 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxxzj\" (UniqueName: \"kubernetes.io/projected/75e0da4f-f558-49ae-969f-14dcb906ebe6-kube-api-access-vxxzj\") pod \"tuned-h2k5h\" (UID: \"75e0da4f-f558-49ae-969f-14dcb906ebe6\") " pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.342642 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.342598 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxhc2\" (UniqueName: \"kubernetes.io/projected/d3473130-eefa-4dfb-82b6-7b464489959f-kube-api-access-wxhc2\") pod \"multus-8mtj2\" (UID: \"d3473130-eefa-4dfb-82b6-7b464489959f\") " pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.343008 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.342985 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6z2z\" (UniqueName: \"kubernetes.io/projected/91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b-kube-api-access-f6z2z\") pod \"node-ca-99vc7\" (UID: \"91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b\") " pod="openshift-image-registry/node-ca-99vc7" Jul 22 02:57:53.344246 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.344157 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zlmt\" (UniqueName: \"kubernetes.io/projected/e9a37af0-c577-48af-9af1-c6089fb43ac2-kube-api-access-5zlmt\") pod \"ovnkube-node-zptvl\" (UID: \"e9a37af0-c577-48af-9af1-c6089fb43ac2\") " pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.344824 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.344781 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk6rr\" (UniqueName: \"kubernetes.io/projected/308ab8f1-8fb2-4833-b583-b22cf8e3cddd-kube-api-access-dk6rr\") pod \"node-resolver-7t98c\" (UID: \"308ab8f1-8fb2-4833-b583-b22cf8e3cddd\") " pod="openshift-dns/node-resolver-7t98c" Jul 22 02:57:53.344824 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.344799 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4frm\" (UniqueName: \"kubernetes.io/projected/31d06428-0968-400f-8b93-86bacf6ec123-kube-api-access-z4frm\") pod \"network-metrics-daemon-mlxzc\" (UID: \"31d06428-0968-400f-8b93-86bacf6ec123\") " pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:57:53.367592 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.367531 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-55.ec2.internal" event={"ID":"671074e8de9b7c15ed7884f108ed8b26","Type":"ContainerStarted","Data":"c09d34657d761510c92eb1f4e3879db9cc3219b56671bc66723b13017eeda6d2"} Jul 22 02:57:53.369926 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.369900 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal" event={"ID":"1bb2de6367befd29ddcad04f7dc49a28","Type":"ContainerStarted","Data":"ec9c85e81bd962200fd542303b42cd98e557feffbb0708c5a889659f3525e889"} Jul 22 02:57:53.484907 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.484808 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" Jul 22 02:57:53.493392 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.493364 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8mtj2" Jul 22 02:57:53.502402 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.502374 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7t98c" Jul 22 02:57:53.512392 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.512368 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ch9w2" Jul 22 02:57:53.520807 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.520784 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:57:53.528546 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.528523 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qk5x9" Jul 22 02:57:53.539383 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.539361 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-t8ln6" Jul 22 02:57:53.546765 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.546746 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" Jul 22 02:57:53.554494 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.554475 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-99vc7" Jul 22 02:57:53.826775 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.826687 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tw9c\" (UID: \"183fb194-f313-4528-bf1b-c457f3174d9d\") " pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:57:53.826947 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.826777 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs\") pod \"network-metrics-daemon-mlxzc\" (UID: \"31d06428-0968-400f-8b93-86bacf6ec123\") " pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:57:53.826947 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.826818 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:57:53.826947 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.826895 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:57:53.826947 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.826904 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret podName:183fb194-f313-4528-bf1b-c457f3174d9d nodeName:}" failed. No retries permitted until 2026-07-22 02:57:54.826884506 +0000 UTC m=+4.018078808 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret") pod "global-pull-secret-syncer-9tw9c" (UID: "183fb194-f313-4528-bf1b-c457f3174d9d") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:57:53.826947 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.826930 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs podName:31d06428-0968-400f-8b93-86bacf6ec123 nodeName:}" failed. No retries permitted until 2026-07-22 02:57:54.826919103 +0000 UTC m=+4.018113387 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs") pod "network-metrics-daemon-mlxzc" (UID: "31d06428-0968-400f-8b93-86bacf6ec123") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:57:53.927508 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:53.927472 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xwcm\" (UniqueName: \"kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm\") pod \"network-check-target-szjw4\" (UID: \"5588cb65-fb3d-4955-98d4-5f5d5ad5234c\") " pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:57:53.927688 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.927671 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:57:53.927728 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.927693 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:57:53.927728 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.927705 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5xwcm for pod openshift-network-diagnostics/network-check-target-szjw4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:57:53.927862 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:53.927771 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm podName:5588cb65-fb3d-4955-98d4-5f5d5ad5234c nodeName:}" failed. No retries permitted until 2026-07-22 02:57:54.927753775 +0000 UTC m=+4.118948058 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5xwcm" (UniqueName: "kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm") pod "network-check-target-szjw4" (UID: "5588cb65-fb3d-4955-98d4-5f5d5ad5234c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:57:54.248461 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:54.248340 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 02:52:52 +0000 UTC" deadline="2027-12-16 16:05:21.956853422 +0000 UTC" Jul 22 02:57:54.248461 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:54.248403 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12301h7m27.708453998s" Jul 22 02:57:54.361169 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:54.361140 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:57:54.361364 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:54.361264 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:57:54.361364 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:54.361307 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:57:54.361485 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:54.361376 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:57:54.441530 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:54.441499 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0625b87a_9139_4d7e_9a02_78c535b9473a.slice/crio-e1d779f08aefa3fdf6f3eff7079016d929c4d5d85ca81a65ba7cee8ff1c6779f WatchSource:0}: Error finding container e1d779f08aefa3fdf6f3eff7079016d929c4d5d85ca81a65ba7cee8ff1c6779f: Status 404 returned error can't find the container with id e1d779f08aefa3fdf6f3eff7079016d929c4d5d85ca81a65ba7cee8ff1c6779f Jul 22 02:57:54.447146 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:54.447127 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9a37af0_c577_48af_9af1_c6089fb43ac2.slice/crio-e2889dad70e438b91fb3fd45ea3b35b542f442c85aae100fedc2b1ed5b1fd527 WatchSource:0}: Error finding container e2889dad70e438b91fb3fd45ea3b35b542f442c85aae100fedc2b1ed5b1fd527: Status 404 returned error can't find the container with id e2889dad70e438b91fb3fd45ea3b35b542f442c85aae100fedc2b1ed5b1fd527 Jul 22 02:57:54.448865 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:54.448802 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae2fd4b_4b95_4e25_832f_8e2599d50b8d.slice/crio-0bbeaf53fc900f7dee0545d5addc1359c527eb798de9b5d2e0cfb3bf023af2a5 WatchSource:0}: Error finding container 0bbeaf53fc900f7dee0545d5addc1359c527eb798de9b5d2e0cfb3bf023af2a5: Status 404 returned error can't find the container with id 0bbeaf53fc900f7dee0545d5addc1359c527eb798de9b5d2e0cfb3bf023af2a5 Jul 22 02:57:54.449362 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:54.449341 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode570fdb6_6eec_4ee0_8ddf_f53f5c9f5539.slice/crio-1713771574bf9a2d4efbad1dd70f12c4935f7e13e073193ef34d73b460b6ad4e WatchSource:0}: Error finding container 1713771574bf9a2d4efbad1dd70f12c4935f7e13e073193ef34d73b460b6ad4e: Status 404 returned error can't find the container with id 1713771574bf9a2d4efbad1dd70f12c4935f7e13e073193ef34d73b460b6ad4e Jul 22 02:57:54.449943 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:54.449919 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa320f4_3fb1_4b07_b021_2f00bcff4f88.slice/crio-8d7aa4c649421e0a2538f6359a5a4942fdc33a0ba217ce12f3cd4eb9bc1aa622 WatchSource:0}: Error finding container 8d7aa4c649421e0a2538f6359a5a4942fdc33a0ba217ce12f3cd4eb9bc1aa622: Status 404 returned error can't find the container with id 8d7aa4c649421e0a2538f6359a5a4942fdc33a0ba217ce12f3cd4eb9bc1aa622 Jul 22 02:57:54.451451 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:54.451426 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3473130_eefa_4dfb_82b6_7b464489959f.slice/crio-05d6a75452968f850d07bb24def7fbe00e19c8e4c6a247ee5d0a9a60fadc82f3 WatchSource:0}: Error finding container 05d6a75452968f850d07bb24def7fbe00e19c8e4c6a247ee5d0a9a60fadc82f3: Status 404 returned error can't find the container with id 05d6a75452968f850d07bb24def7fbe00e19c8e4c6a247ee5d0a9a60fadc82f3 Jul 22 02:57:54.452611 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:54.452242 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75e0da4f_f558_49ae_969f_14dcb906ebe6.slice/crio-001d873151ebdda07c0e3c94108b41a04ee584ec7f430e54e3d5cc9d945421ad WatchSource:0}: Error finding container 001d873151ebdda07c0e3c94108b41a04ee584ec7f430e54e3d5cc9d945421ad: Status 404 returned error can't find the container with id 001d873151ebdda07c0e3c94108b41a04ee584ec7f430e54e3d5cc9d945421ad Jul 22 02:57:54.453152 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:54.453074 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a9bdc2_87c0_47f1_93d7_1f6ba5ba524b.slice/crio-0cba207e672d57c1486b7502d864796aaccfc86e4ffb67b72da9f2f9520bb3e0 WatchSource:0}: Error finding container 0cba207e672d57c1486b7502d864796aaccfc86e4ffb67b72da9f2f9520bb3e0: Status 404 returned error can't find the container with id 0cba207e672d57c1486b7502d864796aaccfc86e4ffb67b72da9f2f9520bb3e0 Jul 22 02:57:54.473974 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:57:54.473939 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod308ab8f1_8fb2_4833_b583_b22cf8e3cddd.slice/crio-f3ac3a6dab267fb8bd3512951497daee6984379e3dbc163e113478ccccc5371f WatchSource:0}: Error finding container f3ac3a6dab267fb8bd3512951497daee6984379e3dbc163e113478ccccc5371f: Status 404 returned error can't find the container with id f3ac3a6dab267fb8bd3512951497daee6984379e3dbc163e113478ccccc5371f Jul 22 02:57:54.835071 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:54.834809 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs\") pod \"network-metrics-daemon-mlxzc\" (UID: \"31d06428-0968-400f-8b93-86bacf6ec123\") " pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:57:54.835071 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:54.835016 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tw9c\" (UID: \"183fb194-f313-4528-bf1b-c457f3174d9d\") " pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:57:54.835071 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:54.834964 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:57:54.835286 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:54.835095 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:57:54.835286 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:54.835119 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs podName:31d06428-0968-400f-8b93-86bacf6ec123 nodeName:}" failed. No retries permitted until 2026-07-22 02:57:56.83510407 +0000 UTC m=+6.026298352 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs") pod "network-metrics-daemon-mlxzc" (UID: "31d06428-0968-400f-8b93-86bacf6ec123") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:57:54.835286 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:54.835133 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret podName:183fb194-f313-4528-bf1b-c457f3174d9d nodeName:}" failed. No retries permitted until 2026-07-22 02:57:56.835126913 +0000 UTC m=+6.026321194 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret") pod "global-pull-secret-syncer-9tw9c" (UID: "183fb194-f313-4528-bf1b-c457f3174d9d") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:57:54.936181 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:54.936140 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xwcm\" (UniqueName: \"kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm\") pod \"network-check-target-szjw4\" (UID: \"5588cb65-fb3d-4955-98d4-5f5d5ad5234c\") " pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:57:54.936374 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:54.936353 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:57:54.936443 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:54.936383 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:57:54.936443 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:54.936399 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5xwcm for pod openshift-network-diagnostics/network-check-target-szjw4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:57:54.936552 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:54.936467 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm podName:5588cb65-fb3d-4955-98d4-5f5d5ad5234c nodeName:}" failed. No retries permitted until 2026-07-22 02:57:56.936447665 +0000 UTC m=+6.127641946 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5xwcm" (UniqueName: "kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm") pod "network-check-target-szjw4" (UID: "5588cb65-fb3d-4955-98d4-5f5d5ad5234c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:57:55.367963 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:55.367390 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:57:55.367963 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:55.367535 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:57:55.377724 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:55.377611 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" event={"ID":"e9a37af0-c577-48af-9af1-c6089fb43ac2","Type":"ContainerStarted","Data":"e2889dad70e438b91fb3fd45ea3b35b542f442c85aae100fedc2b1ed5b1fd527"} Jul 22 02:57:55.380792 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:55.380327 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qk5x9" event={"ID":"0625b87a-9139-4d7e-9a02-78c535b9473a","Type":"ContainerStarted","Data":"e1d779f08aefa3fdf6f3eff7079016d929c4d5d85ca81a65ba7cee8ff1c6779f"} Jul 22 02:57:55.398767 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:55.397998 2580 generic.go:356] "Generic (PLEG): container finished" podID="1bb2de6367befd29ddcad04f7dc49a28" containerID="da8ecd2fd4da73f8e96ccf4a01a1b47beeff261d5dfbef95ca8918094a02b2b2" exitCode=0 Jul 22 02:57:55.398767 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:55.398343 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal" event={"ID":"1bb2de6367befd29ddcad04f7dc49a28","Type":"ContainerDied","Data":"da8ecd2fd4da73f8e96ccf4a01a1b47beeff261d5dfbef95ca8918094a02b2b2"} Jul 22 02:57:55.417371 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:55.417336 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7t98c" event={"ID":"308ab8f1-8fb2-4833-b583-b22cf8e3cddd","Type":"ContainerStarted","Data":"f3ac3a6dab267fb8bd3512951497daee6984379e3dbc163e113478ccccc5371f"} Jul 22 02:57:55.420662 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:55.420562 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t8ln6" event={"ID":"faa320f4-3fb1-4b07-b021-2f00bcff4f88","Type":"ContainerStarted","Data":"8d7aa4c649421e0a2538f6359a5a4942fdc33a0ba217ce12f3cd4eb9bc1aa622"} Jul 22 02:57:55.436383 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:55.436348 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" event={"ID":"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539","Type":"ContainerStarted","Data":"1713771574bf9a2d4efbad1dd70f12c4935f7e13e073193ef34d73b460b6ad4e"} Jul 22 02:57:55.448323 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:55.448292 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8mtj2" event={"ID":"d3473130-eefa-4dfb-82b6-7b464489959f","Type":"ContainerStarted","Data":"05d6a75452968f850d07bb24def7fbe00e19c8e4c6a247ee5d0a9a60fadc82f3"} Jul 22 02:57:55.463098 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:55.463059 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ch9w2" event={"ID":"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d","Type":"ContainerStarted","Data":"0bbeaf53fc900f7dee0545d5addc1359c527eb798de9b5d2e0cfb3bf023af2a5"} Jul 22 02:57:55.472659 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:55.472618 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-55.ec2.internal" event={"ID":"671074e8de9b7c15ed7884f108ed8b26","Type":"ContainerStarted","Data":"c68dd4c057cd8d439f06f03b89a8a2b715f08946a0a282653a0814696a7b1828"} Jul 22 02:57:55.485949 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:55.485602 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-99vc7" event={"ID":"91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b","Type":"ContainerStarted","Data":"0cba207e672d57c1486b7502d864796aaccfc86e4ffb67b72da9f2f9520bb3e0"} Jul 22 02:57:55.492018 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:55.491492 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-55.ec2.internal" podStartSLOduration=3.491472489 podStartE2EDuration="3.491472489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:57:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:57:55.490858453 +0000 UTC m=+4.682052758" watchObservedRunningTime="2026-07-22 02:57:55.491472489 +0000 UTC m=+4.682666793" Jul 22 02:57:55.495144 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:55.495079 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" event={"ID":"75e0da4f-f558-49ae-969f-14dcb906ebe6","Type":"ContainerStarted","Data":"001d873151ebdda07c0e3c94108b41a04ee584ec7f430e54e3d5cc9d945421ad"} Jul 22 02:57:56.361024 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:56.360992 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:57:56.361207 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:56.361124 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:57:56.361636 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:56.361436 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:57:56.361636 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:56.361547 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:57:56.520519 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:56.520474 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal" event={"ID":"1bb2de6367befd29ddcad04f7dc49a28","Type":"ContainerStarted","Data":"7dedc83d1e13b172e80a0e1456c5410e8a577a2a15dbfaf876615707ba45da99"} Jul 22 02:57:56.537531 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:56.537455 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-55.ec2.internal" podStartSLOduration=4.537437377 podStartE2EDuration="4.537437377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:57:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:57:56.536733169 +0000 UTC m=+5.727927497" watchObservedRunningTime="2026-07-22 02:57:56.537437377 +0000 UTC m=+5.728631682" Jul 22 02:57:56.858386 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:56.858303 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs\") pod \"network-metrics-daemon-mlxzc\" (UID: \"31d06428-0968-400f-8b93-86bacf6ec123\") " pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:57:56.858386 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:56.858364 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tw9c\" (UID: \"183fb194-f313-4528-bf1b-c457f3174d9d\") " pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:57:56.858594 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:56.858505 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:57:56.858594 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:56.858569 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret podName:183fb194-f313-4528-bf1b-c457f3174d9d nodeName:}" failed. No retries permitted until 2026-07-22 02:58:00.858551107 +0000 UTC m=+10.049745395 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret") pod "global-pull-secret-syncer-9tw9c" (UID: "183fb194-f313-4528-bf1b-c457f3174d9d") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:57:56.858990 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:56.858968 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:57:56.859063 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:56.859026 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs podName:31d06428-0968-400f-8b93-86bacf6ec123 nodeName:}" failed. No retries permitted until 2026-07-22 02:58:00.85900951 +0000 UTC m=+10.050203797 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs") pod "network-metrics-daemon-mlxzc" (UID: "31d06428-0968-400f-8b93-86bacf6ec123") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:57:56.959238 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:56.959199 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xwcm\" (UniqueName: \"kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm\") pod \"network-check-target-szjw4\" (UID: \"5588cb65-fb3d-4955-98d4-5f5d5ad5234c\") " pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:57:56.959415 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:56.959370 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:57:56.959415 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:56.959390 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:57:56.959415 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:56.959403 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5xwcm for pod openshift-network-diagnostics/network-check-target-szjw4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:57:56.959572 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:56.959462 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm podName:5588cb65-fb3d-4955-98d4-5f5d5ad5234c nodeName:}" failed. No retries permitted until 2026-07-22 02:58:00.959442454 +0000 UTC m=+10.150636741 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5xwcm" (UniqueName: "kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm") pod "network-check-target-szjw4" (UID: "5588cb65-fb3d-4955-98d4-5f5d5ad5234c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:57:57.364382 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:57.363880 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:57:57.364382 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:57.364017 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:57:58.361523 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:58.360858 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:57:58.361523 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:58.360992 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:57:58.361523 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:58.361388 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:57:58.361523 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:58.361478 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:57:59.361355 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:57:59.361325 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:57:59.361526 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:57:59.361467 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:58:00.361650 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:00.361615 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:00.362200 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:00.361748 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:58:00.362200 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:00.361801 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:00.362200 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:00.361937 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:58:00.893248 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:00.892490 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tw9c\" (UID: \"183fb194-f313-4528-bf1b-c457f3174d9d\") " pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:00.893248 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:00.892588 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs\") pod \"network-metrics-daemon-mlxzc\" (UID: \"31d06428-0968-400f-8b93-86bacf6ec123\") " pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:00.893248 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:00.892710 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:58:00.893248 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:00.892772 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs podName:31d06428-0968-400f-8b93-86bacf6ec123 nodeName:}" failed. No retries permitted until 2026-07-22 02:58:08.892751588 +0000 UTC m=+18.083945877 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs") pod "network-metrics-daemon-mlxzc" (UID: "31d06428-0968-400f-8b93-86bacf6ec123") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:58:00.893248 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:00.893161 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:58:00.893248 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:00.893217 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret podName:183fb194-f313-4528-bf1b-c457f3174d9d nodeName:}" failed. No retries permitted until 2026-07-22 02:58:08.893193619 +0000 UTC m=+18.084387917 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret") pod "global-pull-secret-syncer-9tw9c" (UID: "183fb194-f313-4528-bf1b-c457f3174d9d") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:58:00.993703 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:00.993665 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xwcm\" (UniqueName: \"kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm\") pod \"network-check-target-szjw4\" (UID: \"5588cb65-fb3d-4955-98d4-5f5d5ad5234c\") " pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:00.993899 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:00.993844 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:58:00.993899 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:00.993863 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:58:00.993899 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:00.993874 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5xwcm for pod openshift-network-diagnostics/network-check-target-szjw4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:58:00.994071 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:00.993933 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm podName:5588cb65-fb3d-4955-98d4-5f5d5ad5234c nodeName:}" failed. No retries permitted until 2026-07-22 02:58:08.993912208 +0000 UTC m=+18.185106492 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5xwcm" (UniqueName: "kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm") pod "network-check-target-szjw4" (UID: "5588cb65-fb3d-4955-98d4-5f5d5ad5234c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:58:01.362311 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:01.362098 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:01.362311 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:01.362241 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:58:02.361065 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:02.361031 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:02.361241 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:02.361083 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:02.361241 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:02.361176 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:58:02.361362 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:02.361299 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:58:03.362031 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:03.361599 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:03.362031 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:03.361758 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:58:04.360768 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:04.360733 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:04.360960 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:04.360748 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:04.360960 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:04.360867 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:58:04.361076 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:04.360959 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:58:05.361182 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:05.361144 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:05.361679 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:05.361281 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:58:06.361633 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:06.361598 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:06.362081 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:06.361728 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:58:06.362081 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:06.361780 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:06.362081 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:06.361920 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:58:07.361552 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:07.361509 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:07.361722 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:07.361640 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:58:08.360774 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:08.360740 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:08.360774 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:08.360774 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:08.361058 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:08.360889 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:58:08.361058 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:08.361005 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:58:08.949204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:08.949176 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs\") pod \"network-metrics-daemon-mlxzc\" (UID: \"31d06428-0968-400f-8b93-86bacf6ec123\") " pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:08.949643 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:08.949222 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tw9c\" (UID: \"183fb194-f313-4528-bf1b-c457f3174d9d\") " pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:08.949643 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:08.949356 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:58:08.949643 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:08.949426 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret podName:183fb194-f313-4528-bf1b-c457f3174d9d nodeName:}" failed. No retries permitted until 2026-07-22 02:58:24.949405024 +0000 UTC m=+34.140599305 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret") pod "global-pull-secret-syncer-9tw9c" (UID: "183fb194-f313-4528-bf1b-c457f3174d9d") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:58:08.949643 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:08.949355 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:58:08.949643 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:08.949516 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs podName:31d06428-0968-400f-8b93-86bacf6ec123 nodeName:}" failed. No retries permitted until 2026-07-22 02:58:24.949495068 +0000 UTC m=+34.140689353 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs") pod "network-metrics-daemon-mlxzc" (UID: "31d06428-0968-400f-8b93-86bacf6ec123") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:58:09.049850 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:09.049811 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xwcm\" (UniqueName: \"kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm\") pod \"network-check-target-szjw4\" (UID: \"5588cb65-fb3d-4955-98d4-5f5d5ad5234c\") " pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:09.050014 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:09.049961 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:58:09.050014 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:09.049984 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:58:09.050014 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:09.050004 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5xwcm for pod openshift-network-diagnostics/network-check-target-szjw4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:58:09.050137 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:09.050056 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm podName:5588cb65-fb3d-4955-98d4-5f5d5ad5234c nodeName:}" failed. No retries permitted until 2026-07-22 02:58:25.050040347 +0000 UTC m=+34.241234628 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5xwcm" (UniqueName: "kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm") pod "network-check-target-szjw4" (UID: "5588cb65-fb3d-4955-98d4-5f5d5ad5234c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:58:09.361474 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:09.361395 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:09.361620 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:09.361541 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:58:10.361473 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:10.361434 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:10.361903 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:10.361434 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:10.361903 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:10.361548 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:58:10.361903 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:10.361651 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:58:11.362747 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:11.362709 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:11.363218 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:11.362851 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:58:12.360803 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:12.360765 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:12.360985 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:12.360810 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:12.360985 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:12.360899 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:58:12.361119 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:12.361022 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:58:13.361367 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:13.360925 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:13.361367 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:13.361065 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:58:13.553268 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:13.553046 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ch9w2" event={"ID":"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d","Type":"ContainerStarted","Data":"266bbd3620825ba16f38f3720683cb1ba986ea03a8303818f07737450c5d882a"} Jul 22 02:58:13.555167 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:13.555117 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-99vc7" event={"ID":"91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b","Type":"ContainerStarted","Data":"2c95d29af9e7d408b9fe05e084a05cc65458716eb3da97da44782b73cf0acff7"} Jul 22 02:58:13.556434 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:13.556362 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" event={"ID":"75e0da4f-f558-49ae-969f-14dcb906ebe6","Type":"ContainerStarted","Data":"dce30bc4370b5b1ba2828a2a07365dcda9f844469bb3559e8dd6792934c3d5c6"} Jul 22 02:58:13.557903 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:13.557853 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" event={"ID":"e9a37af0-c577-48af-9af1-c6089fb43ac2","Type":"ContainerStarted","Data":"dcd834489a27aa50fddd94dbc2142ede013f50528b1d8a810eb2c84117c0ac62"} Jul 22 02:58:13.559282 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:13.559263 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qk5x9" event={"ID":"0625b87a-9139-4d7e-9a02-78c535b9473a","Type":"ContainerStarted","Data":"69ef1c15bf3ea39aa7ef813d9f722a5e5ea08a90861252427fb52942e20d4836"} Jul 22 02:58:13.560551 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:13.560532 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7t98c" event={"ID":"308ab8f1-8fb2-4833-b583-b22cf8e3cddd","Type":"ContainerStarted","Data":"662a4e4cd42c9ec9b792b5b2b9eca920e22bfae1fad3d9f8fb91601cd9069728"} Jul 22 02:58:13.562009 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:13.561980 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" event={"ID":"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539","Type":"ContainerStarted","Data":"510f7dfff4f424eb65e2051cd15ad5857fa45d61bd66c41d5e37842fc5766336"} Jul 22 02:58:13.589751 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:13.589688 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7t98c" podStartSLOduration=3.7533077759999998 podStartE2EDuration="22.589671518s" totalImagesPullingTime="18.836363742s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:58:13.58944273 +0000 UTC m=+22.780637027" watchObservedRunningTime="2026-07-22 02:58:13.589671518 +0000 UTC m=+22.780865825" Jul 22 02:58:13.603250 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:13.603204 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qk5x9" podStartSLOduration=12.219238263 podStartE2EDuration="22.603185029s" totalImagesPullingTime="10.383946766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:58:13.602716074 +0000 UTC m=+22.793910388" watchObservedRunningTime="2026-07-22 02:58:13.603185029 +0000 UTC m=+22.794379334" Jul 22 02:58:13.617776 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:13.617741 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-h2k5h" podStartSLOduration=3.776290923 podStartE2EDuration="22.617726511s" totalImagesPullingTime="18.841435588s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:58:13.617471381 +0000 UTC m=+22.808665685" watchObservedRunningTime="2026-07-22 02:58:13.617726511 +0000 UTC m=+22.808920813" Jul 22 02:58:14.361111 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:14.360942 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:14.361229 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:14.360942 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:14.361229 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:14.361185 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:58:14.361312 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:14.361255 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:58:14.566756 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:14.566731 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 02:58:14.567557 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:14.567064 2580 generic.go:356] "Generic (PLEG): container finished" podID="e9a37af0-c577-48af-9af1-c6089fb43ac2" containerID="d7c320afbfab410d9720b022be400462ac3353d93225a50c4117f39bfdbd853a" exitCode=1 Jul 22 02:58:14.567557 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:14.567138 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" event={"ID":"e9a37af0-c577-48af-9af1-c6089fb43ac2","Type":"ContainerStarted","Data":"18084132813e88adbd9285d4e4d09a8664ac086c2b8f4411f26b5323e5926c8f"} Jul 22 02:58:14.567557 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:14.567250 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" event={"ID":"e9a37af0-c577-48af-9af1-c6089fb43ac2","Type":"ContainerStarted","Data":"42954baeb8c71ba6daa432a8f54bdc92588c73814215dcb9640dd53cc1ddd7af"} Jul 22 02:58:14.567557 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:14.567295 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" event={"ID":"e9a37af0-c577-48af-9af1-c6089fb43ac2","Type":"ContainerStarted","Data":"ec6292367837811b6a8b168a213f1ab037fc722654bd2df1b65405f1e9d2122a"} Jul 22 02:58:14.567557 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:14.567338 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" event={"ID":"e9a37af0-c577-48af-9af1-c6089fb43ac2","Type":"ContainerStarted","Data":"18c2239395976d41384feedb35e6b0474c5e0dfeb4f2238f615731f5360e8a56"} Jul 22 02:58:14.567557 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:14.567383 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" event={"ID":"e9a37af0-c577-48af-9af1-c6089fb43ac2","Type":"ContainerDied","Data":"d7c320afbfab410d9720b022be400462ac3353d93225a50c4117f39bfdbd853a"} Jul 22 02:58:14.568528 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:14.568506 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8mtj2" event={"ID":"d3473130-eefa-4dfb-82b6-7b464489959f","Type":"ContainerStarted","Data":"e890768989c5e46e4458daa64d224fef69aa0330cee3082632c9ead387255d61"} Jul 22 02:58:14.569924 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:14.569901 2580 generic.go:356] "Generic (PLEG): container finished" podID="0ae2fd4b-4b95-4e25-832f-8e2599d50b8d" containerID="266bbd3620825ba16f38f3720683cb1ba986ea03a8303818f07737450c5d882a" exitCode=0 Jul 22 02:58:14.570031 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:14.569972 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ch9w2" event={"ID":"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d","Type":"ContainerDied","Data":"266bbd3620825ba16f38f3720683cb1ba986ea03a8303818f07737450c5d882a"} Jul 22 02:58:14.585939 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:14.585879 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-99vc7" podStartSLOduration=4.762751971 podStartE2EDuration="23.58586774s" totalImagesPullingTime="18.823115769s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:58:13.636593452 +0000 UTC m=+22.827787776" watchObservedRunningTime="2026-07-22 02:58:14.58586774 +0000 UTC m=+23.777062043" Jul 22 02:58:14.586029 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:14.585938 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8mtj2" podStartSLOduration=4.480708338 podStartE2EDuration="23.585935402s" totalImagesPullingTime="19.105227064s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:58:14.585037735 +0000 UTC m=+23.776232038" watchObservedRunningTime="2026-07-22 02:58:14.585935402 +0000 UTC m=+23.777129704" Jul 22 02:58:14.829684 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:14.829656 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 02:58:15.287299 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:15.287187 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T02:58:14.829672934Z","UUID":"c342e814-bd14-42c6-badb-870fca72b08d","Handler":null,"Name":"","Endpoint":""} Jul 22 02:58:15.288913 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:15.288888 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 02:58:15.288913 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:15.288929 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 02:58:15.361228 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:15.361197 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:15.361395 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:15.361334 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:58:15.574337 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:15.574252 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-t8ln6" event={"ID":"faa320f4-3fb1-4b07-b021-2f00bcff4f88","Type":"ContainerStarted","Data":"48003d621886efb6933257c88913b895c3082b11f0223d7c93ec606d28102691"} Jul 22 02:58:15.576628 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:15.576560 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" event={"ID":"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539","Type":"ContainerStarted","Data":"ff94219f6a28c2cde716d766b0d4575556eba9a4e654e7e68603bbfe65578d5c"} Jul 22 02:58:15.589382 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:15.589327 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-t8ln6" podStartSLOduration=5.748945996 podStartE2EDuration="24.589307705s" totalImagesPullingTime="18.840361709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:58:15.588963149 +0000 UTC m=+24.780157453" watchObservedRunningTime="2026-07-22 02:58:15.589307705 +0000 UTC m=+24.780502011" Jul 22 02:58:16.361873 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:16.361690 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:16.362030 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:16.361702 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:16.362030 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:16.361942 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:58:16.362102 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:16.362028 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:58:16.582399 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:16.582322 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 02:58:16.582866 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:16.582805 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" event={"ID":"e9a37af0-c577-48af-9af1-c6089fb43ac2","Type":"ContainerStarted","Data":"dbc4b42dbffcbcaa14a5e256b827ef8b32e50a461d09788c6c2d63db71b1b10d"} Jul 22 02:58:16.585060 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:16.585033 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" event={"ID":"e570fdb6-6eec-4ee0-8ddf-f53f5c9f5539","Type":"ContainerStarted","Data":"17bd80632e5102cfc0281eca04c25e42f3613cf595b801e7960f1a750c8fb092"} Jul 22 02:58:16.603284 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:16.603246 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d2bzj" podStartSLOduration=4.527490413 podStartE2EDuration="25.603234945s" totalImagesPullingTime="21.075744532s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:57:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:58:16.601825538 +0000 UTC m=+25.793019852" watchObservedRunningTime="2026-07-22 02:58:16.603234945 +0000 UTC m=+25.794429281" Jul 22 02:58:17.128882 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:17.128845 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qk5x9" Jul 22 02:58:17.129922 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:17.129904 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qk5x9" Jul 22 02:58:17.361102 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:17.361040 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:17.361337 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:17.361176 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:58:18.361014 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:18.360984 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:18.361596 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:18.361088 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:58:18.361596 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:18.361152 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:18.361596 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:18.361248 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:58:19.361336 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:19.361144 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:19.361783 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:19.361401 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:58:19.595752 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:19.595722 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 02:58:19.599168 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:19.599138 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" event={"ID":"e9a37af0-c577-48af-9af1-c6089fb43ac2","Type":"ContainerStarted","Data":"9ecce3b43b93398a39f0c1489f56b8227febe3165162760a1d447bf76788cc96"} Jul 22 02:58:19.599540 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:19.599522 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:58:19.599816 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:19.599797 2580 scope.go:117] "RemoveContainer" containerID="d7c320afbfab410d9720b022be400462ac3353d93225a50c4117f39bfdbd853a" Jul 22 02:58:19.622581 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:19.622564 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:58:20.361593 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:20.361567 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:20.361970 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:20.361567 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:20.361970 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:20.361663 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:58:20.361970 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:20.361729 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:58:20.603523 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:20.603490 2580 generic.go:356] "Generic (PLEG): container finished" podID="0ae2fd4b-4b95-4e25-832f-8e2599d50b8d" containerID="c9a662c918b33a37dec1b6014cc0505fcadbecb5101a73d75d7d2b21799f8840" exitCode=0 Jul 22 02:58:20.603731 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:20.603572 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ch9w2" event={"ID":"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d","Type":"ContainerDied","Data":"c9a662c918b33a37dec1b6014cc0505fcadbecb5101a73d75d7d2b21799f8840"} Jul 22 02:58:20.607500 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:20.607477 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 02:58:20.607849 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:20.607818 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" event={"ID":"e9a37af0-c577-48af-9af1-c6089fb43ac2","Type":"ContainerStarted","Data":"97379390e3584530af86c474b4ba478e6f04e773e20eca800092cdf7c47a3dd2"} Jul 22 02:58:20.608101 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:20.608088 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:58:20.608156 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:20.608146 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:58:20.622807 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:20.622786 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:58:20.648391 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:20.648351 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" podStartSLOduration=10.727452787 podStartE2EDuration="29.648338552s" totalImagesPullingTime="18.920885765s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:58:20.647723576 +0000 UTC m=+29.838917888" watchObservedRunningTime="2026-07-22 02:58:20.648338552 +0000 UTC m=+29.839532854" Jul 22 02:58:21.031126 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:21.031054 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qk5x9" Jul 22 02:58:21.031290 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:21.031230 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 22 02:58:21.031642 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:21.031619 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qk5x9" Jul 22 02:58:21.362344 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:21.362310 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:21.362699 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:21.362399 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:58:21.440040 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:21.440014 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mlxzc"] Jul 22 02:58:21.442851 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:21.442817 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9tw9c"] Jul 22 02:58:21.442960 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:21.442914 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:21.443028 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:21.443007 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:58:21.457953 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:21.457931 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-szjw4"] Jul 22 02:58:21.458066 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:21.458041 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:21.458176 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:21.458155 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:58:21.610270 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:21.610242 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:21.610738 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:21.610709 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:58:22.613947 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:22.613749 2580 generic.go:356] "Generic (PLEG): container finished" podID="0ae2fd4b-4b95-4e25-832f-8e2599d50b8d" containerID="37fc9b8210835bb6f335f05e83e4425c273d4c02afa92c328afd2313acfe4edc" exitCode=0 Jul 22 02:58:22.614387 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:22.613844 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ch9w2" event={"ID":"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d","Type":"ContainerDied","Data":"37fc9b8210835bb6f335f05e83e4425c273d4c02afa92c328afd2313acfe4edc"} Jul 22 02:58:23.360961 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:23.360931 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:23.361138 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:23.360970 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:23.361138 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:23.360934 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:23.361138 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:23.361033 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:58:23.361138 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:23.361118 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:58:23.361347 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:23.361184 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:58:24.620735 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:24.620701 2580 generic.go:356] "Generic (PLEG): container finished" podID="0ae2fd4b-4b95-4e25-832f-8e2599d50b8d" containerID="d73205246516ede1042072b1f4d7d41bcf93dc08c97ec48b1f08714054d27f63" exitCode=0 Jul 22 02:58:24.621281 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:24.620744 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ch9w2" event={"ID":"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d","Type":"ContainerDied","Data":"d73205246516ede1042072b1f4d7d41bcf93dc08c97ec48b1f08714054d27f63"} Jul 22 02:58:24.966944 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:24.966870 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs\") pod \"network-metrics-daemon-mlxzc\" (UID: \"31d06428-0968-400f-8b93-86bacf6ec123\") " pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:24.967080 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:24.966927 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tw9c\" (UID: \"183fb194-f313-4528-bf1b-c457f3174d9d\") " pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:24.967080 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:24.967019 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:58:24.967080 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:24.967041 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:58:24.967182 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:24.967089 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs podName:31d06428-0968-400f-8b93-86bacf6ec123 nodeName:}" failed. No retries permitted until 2026-07-22 02:58:56.967074156 +0000 UTC m=+66.158268453 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs") pod "network-metrics-daemon-mlxzc" (UID: "31d06428-0968-400f-8b93-86bacf6ec123") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:58:24.967182 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:24.967103 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret podName:183fb194-f313-4528-bf1b-c457f3174d9d nodeName:}" failed. No retries permitted until 2026-07-22 02:58:56.967097479 +0000 UTC m=+66.158291760 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret") pod "global-pull-secret-syncer-9tw9c" (UID: "183fb194-f313-4528-bf1b-c457f3174d9d") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:58:25.067487 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:25.067458 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xwcm\" (UniqueName: \"kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm\") pod \"network-check-target-szjw4\" (UID: \"5588cb65-fb3d-4955-98d4-5f5d5ad5234c\") " pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:25.067623 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:25.067602 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:58:25.067623 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:25.067619 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:58:25.067697 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:25.067628 2580 projected.go:196] Error preparing data for projected volume kube-api-access-5xwcm for pod openshift-network-diagnostics/network-check-target-szjw4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:58:25.067697 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:25.067675 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm podName:5588cb65-fb3d-4955-98d4-5f5d5ad5234c nodeName:}" failed. No retries permitted until 2026-07-22 02:58:57.067660937 +0000 UTC m=+66.258855217 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5xwcm" (UniqueName: "kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm") pod "network-check-target-szjw4" (UID: "5588cb65-fb3d-4955-98d4-5f5d5ad5234c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:58:25.361353 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:25.361325 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:25.361527 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:25.361325 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:25.361527 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:25.361433 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlxzc" podUID="31d06428-0968-400f-8b93-86bacf6ec123" Jul 22 02:58:25.361635 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:25.361521 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-szjw4" podUID="5588cb65-fb3d-4955-98d4-5f5d5ad5234c" Jul 22 02:58:25.361635 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:25.361565 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:25.361740 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:25.361638 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tw9c" podUID="183fb194-f313-4528-bf1b-c457f3174d9d" Jul 22 02:58:27.186279 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.186251 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-55.ec2.internal" event="NodeReady" Jul 22 02:58:27.186728 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.186375 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 02:58:27.219728 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.219660 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-984955c4f-ff8rk"] Jul 22 02:58:27.224423 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.224401 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.226361 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.226339 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 02:58:27.226675 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.226654 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 02:58:27.226946 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.226921 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7vzzs\"" Jul 22 02:58:27.227082 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.226967 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 02:58:27.233156 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.232994 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 02:58:27.237465 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.237441 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b46cz"] Jul 22 02:58:27.240874 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.240854 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-984955c4f-ff8rk"] Jul 22 02:58:27.241052 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.241028 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-r6wl8"] Jul 22 02:58:27.241176 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.240963 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:27.243237 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.243215 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 02:58:27.243330 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.243276 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sn76t\"" Jul 22 02:58:27.243330 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.243303 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 02:58:27.244153 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.244134 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r6wl8" Jul 22 02:58:27.246273 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.246259 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 02:58:27.246507 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.246490 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 02:58:27.246576 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.246544 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-h84cw\"" Jul 22 02:58:27.246616 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.246599 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 02:58:27.249566 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.249549 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b46cz"] Jul 22 02:58:27.262590 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.262567 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r6wl8"] Jul 22 02:58:27.361881 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.361845 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:27.362080 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.361845 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:27.362080 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.362062 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:27.364972 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.364886 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 02:58:27.364972 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.364923 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nztgp\"" Jul 22 02:58:27.365171 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.365118 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 02:58:27.365171 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.365125 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 02:58:27.365277 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.365178 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 02:58:27.365277 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.365252 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sqq4d\"" Jul 22 02:58:27.385021 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.384999 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-installation-pull-secrets\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.385117 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.385030 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-ca-trust-extracted\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.385117 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.385048 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-certificates\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.385117 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.385063 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-bound-sa-token\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.385117 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.385101 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmwkn\" (UniqueName: \"kubernetes.io/projected/7981d361-b9b8-4e56-97ad-928079266fb8-kube-api-access-fmwkn\") pod \"ingress-canary-r6wl8\" (UID: \"7981d361-b9b8-4e56-97ad-928079266fb8\") " pod="openshift-ingress-canary/ingress-canary-r6wl8" Jul 22 02:58:27.385286 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.385153 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:27.385286 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.385198 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/33beac89-d63c-4c71-82db-1790e90a1447-config-volume\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:27.385286 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.385220 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33beac89-d63c-4c71-82db-1790e90a1447-tmp-dir\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:27.385392 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.385289 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rn4r\" (UniqueName: \"kubernetes.io/projected/33beac89-d63c-4c71-82db-1790e90a1447-kube-api-access-9rn4r\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:27.385392 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.385329 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.385392 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.385349 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7981d361-b9b8-4e56-97ad-928079266fb8-cert\") pod \"ingress-canary-r6wl8\" (UID: \"7981d361-b9b8-4e56-97ad-928079266fb8\") " pod="openshift-ingress-canary/ingress-canary-r6wl8" Jul 22 02:58:27.385392 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.385377 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prlgm\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-kube-api-access-prlgm\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.385552 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.385433 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-trusted-ca\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.385552 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.385476 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-image-registry-private-configuration\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.475427 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.475344 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qk5x9_0625b87a-9139-4d7e-9a02-78c535b9473a/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:27.486321 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.486291 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-trusted-ca\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.486453 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.486345 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-image-registry-private-configuration\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.486453 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.486381 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-installation-pull-secrets\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.486453 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.486406 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-ca-trust-extracted\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.486453 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.486432 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-certificates\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.487870 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.487186 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-ca-trust-extracted\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.487870 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.487296 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-bound-sa-token\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.489000 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.488038 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmwkn\" (UniqueName: \"kubernetes.io/projected/7981d361-b9b8-4e56-97ad-928079266fb8-kube-api-access-fmwkn\") pod \"ingress-canary-r6wl8\" (UID: \"7981d361-b9b8-4e56-97ad-928079266fb8\") " pod="openshift-ingress-canary/ingress-canary-r6wl8" Jul 22 02:58:27.489000 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.488133 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:27.489000 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.488295 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/33beac89-d63c-4c71-82db-1790e90a1447-config-volume\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:27.489000 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.488426 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33beac89-d63c-4c71-82db-1790e90a1447-tmp-dir\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:27.489000 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.488585 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-trusted-ca\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.489000 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.488932 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-certificates\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.489337 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:27.489104 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:58:27.489337 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.489260 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33beac89-d63c-4c71-82db-1790e90a1447-tmp-dir\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:27.491552 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.489872 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rn4r\" (UniqueName: \"kubernetes.io/projected/33beac89-d63c-4c71-82db-1790e90a1447-kube-api-access-9rn4r\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:27.491552 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.489917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.491552 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.489944 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7981d361-b9b8-4e56-97ad-928079266fb8-cert\") pod \"ingress-canary-r6wl8\" (UID: \"7981d361-b9b8-4e56-97ad-928079266fb8\") " pod="openshift-ingress-canary/ingress-canary-r6wl8" Jul 22 02:58:27.491552 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.490001 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prlgm\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-kube-api-access-prlgm\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.491552 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:27.490362 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:58:27.491552 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:27.490377 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-984955c4f-ff8rk: secret "image-registry-tls" not found Jul 22 02:58:27.491552 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:27.490441 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls podName:01f1c1c1-3d02-4a21-8468-4c8c30b91c5a nodeName:}" failed. No retries permitted until 2026-07-22 02:58:27.99041574 +0000 UTC m=+37.181610036 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls") pod "image-registry-984955c4f-ff8rk" (UID: "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a") : secret "image-registry-tls" not found Jul 22 02:58:27.491552 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:27.491524 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls podName:33beac89-d63c-4c71-82db-1790e90a1447 nodeName:}" failed. No retries permitted until 2026-07-22 02:58:27.991501912 +0000 UTC m=+37.182696195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls") pod "dns-default-b46cz" (UID: "33beac89-d63c-4c71-82db-1790e90a1447") : secret "dns-default-metrics-tls" not found Jul 22 02:58:27.492351 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.492328 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/33beac89-d63c-4c71-82db-1790e90a1447-config-volume\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:27.492729 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.492642 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-installation-pull-secrets\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.493306 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.493283 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-image-registry-private-configuration\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.493725 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.493704 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7981d361-b9b8-4e56-97ad-928079266fb8-cert\") pod \"ingress-canary-r6wl8\" (UID: \"7981d361-b9b8-4e56-97ad-928079266fb8\") " pod="openshift-ingress-canary/ingress-canary-r6wl8" Jul 22 02:58:27.496743 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.496699 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmwkn\" (UniqueName: \"kubernetes.io/projected/7981d361-b9b8-4e56-97ad-928079266fb8-kube-api-access-fmwkn\") pod \"ingress-canary-r6wl8\" (UID: \"7981d361-b9b8-4e56-97ad-928079266fb8\") " pod="openshift-ingress-canary/ingress-canary-r6wl8" Jul 22 02:58:27.497382 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.497360 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-bound-sa-token\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.497590 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.497569 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rn4r\" (UniqueName: \"kubernetes.io/projected/33beac89-d63c-4c71-82db-1790e90a1447-kube-api-access-9rn4r\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:27.498937 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.498915 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prlgm\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-kube-api-access-prlgm\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.558993 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.558968 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r6wl8" Jul 22 02:58:27.744543 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.744450 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qk5x9_0625b87a-9139-4d7e-9a02-78c535b9473a/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:27.994109 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.994077 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:27.994301 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:27.994152 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:27.994301 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:27.994245 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:58:27.994414 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:27.994311 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:58:27.994414 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:27.994325 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-984955c4f-ff8rk: secret "image-registry-tls" not found Jul 22 02:58:27.994414 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:27.994337 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls podName:33beac89-d63c-4c71-82db-1790e90a1447 nodeName:}" failed. No retries permitted until 2026-07-22 02:58:28.994315214 +0000 UTC m=+38.185509508 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls") pod "dns-default-b46cz" (UID: "33beac89-d63c-4c71-82db-1790e90a1447") : secret "dns-default-metrics-tls" not found Jul 22 02:58:27.994414 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:27.994402 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls podName:01f1c1c1-3d02-4a21-8468-4c8c30b91c5a nodeName:}" failed. No retries permitted until 2026-07-22 02:58:28.994384827 +0000 UTC m=+38.185579112 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls") pod "image-registry-984955c4f-ff8rk" (UID: "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a") : secret "image-registry-tls" not found Jul 22 02:58:28.761890 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:28.761736 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r6wl8"] Jul 22 02:58:28.765336 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:58:28.765314 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7981d361_b9b8_4e56_97ad_928079266fb8.slice/crio-b3d18452c1cb09fe653c3a1f6b9776c0ec9e43b81bb2261aa796c78046f4fabb WatchSource:0}: Error finding container b3d18452c1cb09fe653c3a1f6b9776c0ec9e43b81bb2261aa796c78046f4fabb: Status 404 returned error can't find the container with id b3d18452c1cb09fe653c3a1f6b9776c0ec9e43b81bb2261aa796c78046f4fabb Jul 22 02:58:29.003409 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:29.003379 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:29.003512 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:29.003461 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:29.003551 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:29.003539 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:58:29.003582 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:29.003556 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-984955c4f-ff8rk: secret "image-registry-tls" not found Jul 22 02:58:29.003582 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:29.003556 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:58:29.003641 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:29.003614 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls podName:01f1c1c1-3d02-4a21-8468-4c8c30b91c5a nodeName:}" failed. No retries permitted until 2026-07-22 02:58:31.003598212 +0000 UTC m=+40.194792493 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls") pod "image-registry-984955c4f-ff8rk" (UID: "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a") : secret "image-registry-tls" not found Jul 22 02:58:29.003641 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:29.003629 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls podName:33beac89-d63c-4c71-82db-1790e90a1447 nodeName:}" failed. No retries permitted until 2026-07-22 02:58:31.003623675 +0000 UTC m=+40.194817955 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls") pod "dns-default-b46cz" (UID: "33beac89-d63c-4c71-82db-1790e90a1447") : secret "dns-default-metrics-tls" not found Jul 22 02:58:29.636071 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:29.636035 2580 generic.go:356] "Generic (PLEG): container finished" podID="0ae2fd4b-4b95-4e25-832f-8e2599d50b8d" containerID="52b858a6df2354b5ffa2949014fbaf985531eb753a68c871fc8e5a6482ec56e9" exitCode=0 Jul 22 02:58:29.636244 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:29.636110 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ch9w2" event={"ID":"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d","Type":"ContainerDied","Data":"52b858a6df2354b5ffa2949014fbaf985531eb753a68c871fc8e5a6482ec56e9"} Jul 22 02:58:29.637605 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:29.637585 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r6wl8" event={"ID":"7981d361-b9b8-4e56-97ad-928079266fb8","Type":"ContainerStarted","Data":"b3d18452c1cb09fe653c3a1f6b9776c0ec9e43b81bb2261aa796c78046f4fabb"} Jul 22 02:58:30.643594 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:30.643406 2580 generic.go:356] "Generic (PLEG): container finished" podID="0ae2fd4b-4b95-4e25-832f-8e2599d50b8d" containerID="6ef55737e0fd67e2fea7772bf5d27f47167d3a74e5096340fa3c2d2cc001f9d5" exitCode=0 Jul 22 02:58:30.643594 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:30.643493 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ch9w2" event={"ID":"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d","Type":"ContainerDied","Data":"6ef55737e0fd67e2fea7772bf5d27f47167d3a74e5096340fa3c2d2cc001f9d5"} Jul 22 02:58:30.649906 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:30.649872 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qk5x9_0625b87a-9139-4d7e-9a02-78c535b9473a/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:30.751659 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:30.751613 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qk5x9_0625b87a-9139-4d7e-9a02-78c535b9473a/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:31.021113 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:31.021021 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:31.021113 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:31.021106 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:31.021316 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:31.021194 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:58:31.021316 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:31.021209 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:58:31.021316 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:31.021222 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-984955c4f-ff8rk: secret "image-registry-tls" not found Jul 22 02:58:31.021316 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:31.021262 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls podName:33beac89-d63c-4c71-82db-1790e90a1447 nodeName:}" failed. No retries permitted until 2026-07-22 02:58:35.02124314 +0000 UTC m=+44.212437421 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls") pod "dns-default-b46cz" (UID: "33beac89-d63c-4c71-82db-1790e90a1447") : secret "dns-default-metrics-tls" not found Jul 22 02:58:31.021316 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:31.021282 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls podName:01f1c1c1-3d02-4a21-8468-4c8c30b91c5a nodeName:}" failed. No retries permitted until 2026-07-22 02:58:35.021271115 +0000 UTC m=+44.212465395 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls") pod "image-registry-984955c4f-ff8rk" (UID: "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a") : secret "image-registry-tls" not found Jul 22 02:58:31.649415 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:31.649384 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qk5x9_0625b87a-9139-4d7e-9a02-78c535b9473a/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:31.649739 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:31.649524 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ch9w2" event={"ID":"0ae2fd4b-4b95-4e25-832f-8e2599d50b8d","Type":"ContainerStarted","Data":"616167c433be2605ecec5a80e7d3624fa919f0941c3b0411ada1a897bf57702f"} Jul 22 02:58:31.673490 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:31.673452 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ch9w2" podStartSLOduration=10.657168222 podStartE2EDuration="40.673440979s" totalImagesPullingTime="30.016272757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:57:51 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:58:31.672151131 +0000 UTC m=+40.863345451" watchObservedRunningTime="2026-07-22 02:58:31.673440979 +0000 UTC m=+40.864635280" Jul 22 02:58:31.752463 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:31.752430 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qk5x9_0625b87a-9139-4d7e-9a02-78c535b9473a/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:32.048562 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:32.048531 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qk5x9_0625b87a-9139-4d7e-9a02-78c535b9473a/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:32.121449 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:32.121419 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qk5x9_0625b87a-9139-4d7e-9a02-78c535b9473a/konnectivity-agent/0.log" limit=1024 Jul 22 02:58:32.653274 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:32.653234 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r6wl8" event={"ID":"7981d361-b9b8-4e56-97ad-928079266fb8","Type":"ContainerStarted","Data":"02b612a0686e6e926fb33417ff33066ecdaf765d1ca91f9113604886a7d6d927"} Jul 22 02:58:32.669633 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:32.669591 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-r6wl8" podStartSLOduration=2.840004541 podStartE2EDuration="5.669577572s" totalImagesPullingTime="2.829573031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:58:32.669092524 +0000 UTC m=+41.860286835" watchObservedRunningTime="2026-07-22 02:58:32.669577572 +0000 UTC m=+41.860771902" Jul 22 02:58:35.050527 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:35.050492 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:35.050958 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:35.050547 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:35.050958 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:35.050643 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:58:35.050958 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:35.050705 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls podName:33beac89-d63c-4c71-82db-1790e90a1447 nodeName:}" failed. No retries permitted until 2026-07-22 02:58:43.050687328 +0000 UTC m=+52.241881613 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls") pod "dns-default-b46cz" (UID: "33beac89-d63c-4c71-82db-1790e90a1447") : secret "dns-default-metrics-tls" not found Jul 22 02:58:35.050958 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:35.050646 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:58:35.050958 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:35.050740 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-984955c4f-ff8rk: secret "image-registry-tls" not found Jul 22 02:58:35.050958 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:35.050780 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls podName:01f1c1c1-3d02-4a21-8468-4c8c30b91c5a nodeName:}" failed. No retries permitted until 2026-07-22 02:58:43.050769088 +0000 UTC m=+52.241963369 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls") pod "image-registry-984955c4f-ff8rk" (UID: "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a") : secret "image-registry-tls" not found Jul 22 02:58:43.107889 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:43.107819 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:43.108307 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:43.107914 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:43.108307 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:43.107981 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:58:43.108307 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:43.108015 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:58:43.108307 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:43.108025 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-984955c4f-ff8rk: secret "image-registry-tls" not found Jul 22 02:58:43.108307 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:43.108049 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls podName:33beac89-d63c-4c71-82db-1790e90a1447 nodeName:}" failed. No retries permitted until 2026-07-22 02:58:59.108031833 +0000 UTC m=+68.299226116 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls") pod "dns-default-b46cz" (UID: "33beac89-d63c-4c71-82db-1790e90a1447") : secret "dns-default-metrics-tls" not found Jul 22 02:58:43.108307 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:43.108063 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls podName:01f1c1c1-3d02-4a21-8468-4c8c30b91c5a nodeName:}" failed. No retries permitted until 2026-07-22 02:58:59.108057384 +0000 UTC m=+68.299251664 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls") pod "image-registry-984955c4f-ff8rk" (UID: "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a") : secret "image-registry-tls" not found Jul 22 02:58:44.413115 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.413083 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-frhxj"] Jul 22 02:58:44.433222 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.433196 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-frhxj"] Jul 22 02:58:44.433349 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.433262 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-frhxj" Jul 22 02:58:44.435784 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.435760 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 22 02:58:44.435904 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.435762 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:58:44.435904 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.435798 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-fqhr5\"" Jul 22 02:58:44.531103 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.531073 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h"] Jul 22 02:58:44.549246 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.549223 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-94dxd"] Jul 22 02:58:44.549355 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.549302 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" Jul 22 02:58:44.551983 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.551963 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9b764\"" Jul 22 02:58:44.552843 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.552805 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 02:58:44.553011 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.552806 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 22 02:58:44.553146 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.553127 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 02:58:44.553377 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.553356 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 22 02:58:44.575285 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.575264 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h"] Jul 22 02:58:44.575427 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.575292 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-94dxd"] Jul 22 02:58:44.575427 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.575363 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:58:44.579049 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.579027 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:58:44.579049 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.579042 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 22 02:58:44.579304 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.579288 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 22 02:58:44.579304 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.579300 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-mdk6c\"" Jul 22 02:58:44.579472 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.579352 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 22 02:58:44.585669 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.585653 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 22 02:58:44.619065 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.619045 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw76q\" (UniqueName: \"kubernetes.io/projected/74a3c85e-05c0-4ad6-addb-c70c0996ea10-kube-api-access-pw76q\") pod \"volume-data-source-validator-78cf4c6f6-frhxj\" (UID: \"74a3c85e-05c0-4ad6-addb-c70c0996ea10\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-frhxj" Jul 22 02:58:44.720295 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.720241 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6-serving-cert\") pod \"console-operator-565b7b569d-94dxd\" (UID: \"fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6\") " pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:58:44.720379 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.720295 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw76q\" (UniqueName: \"kubernetes.io/projected/74a3c85e-05c0-4ad6-addb-c70c0996ea10-kube-api-access-pw76q\") pod \"volume-data-source-validator-78cf4c6f6-frhxj\" (UID: \"74a3c85e-05c0-4ad6-addb-c70c0996ea10\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-frhxj" Jul 22 02:58:44.720379 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.720313 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3c7ed38f-86c4-4046-8f3e-516265372dcb-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-tst5h\" (UID: \"3c7ed38f-86c4-4046-8f3e-516265372dcb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" Jul 22 02:58:44.720379 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.720367 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6-trusted-ca\") pod \"console-operator-565b7b569d-94dxd\" (UID: \"fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6\") " pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:58:44.720475 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.720395 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6bgq\" (UniqueName: \"kubernetes.io/projected/fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6-kube-api-access-z6bgq\") pod \"console-operator-565b7b569d-94dxd\" (UID: \"fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6\") " pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:58:44.720475 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.720418 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tst5h\" (UID: \"3c7ed38f-86c4-4046-8f3e-516265372dcb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" Jul 22 02:58:44.720535 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.720496 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8vh8\" (UniqueName: \"kubernetes.io/projected/3c7ed38f-86c4-4046-8f3e-516265372dcb-kube-api-access-n8vh8\") pod \"cluster-monitoring-operator-76cf8966f8-tst5h\" (UID: \"3c7ed38f-86c4-4046-8f3e-516265372dcb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" Jul 22 02:58:44.720567 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.720531 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6-config\") pod \"console-operator-565b7b569d-94dxd\" (UID: \"fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6\") " pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:58:44.735111 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.735085 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw76q\" (UniqueName: \"kubernetes.io/projected/74a3c85e-05c0-4ad6-addb-c70c0996ea10-kube-api-access-pw76q\") pod \"volume-data-source-validator-78cf4c6f6-frhxj\" (UID: \"74a3c85e-05c0-4ad6-addb-c70c0996ea10\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-frhxj" Jul 22 02:58:44.741758 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.741742 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-frhxj" Jul 22 02:58:44.821117 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.821090 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3c7ed38f-86c4-4046-8f3e-516265372dcb-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-tst5h\" (UID: \"3c7ed38f-86c4-4046-8f3e-516265372dcb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" Jul 22 02:58:44.821117 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.821122 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6-trusted-ca\") pod \"console-operator-565b7b569d-94dxd\" (UID: \"fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6\") " pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:58:44.821117 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.821138 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6bgq\" (UniqueName: \"kubernetes.io/projected/fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6-kube-api-access-z6bgq\") pod \"console-operator-565b7b569d-94dxd\" (UID: \"fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6\") " pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:58:44.821117 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.821171 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tst5h\" (UID: \"3c7ed38f-86c4-4046-8f3e-516265372dcb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" Jul 22 02:58:44.821117 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.821237 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8vh8\" (UniqueName: \"kubernetes.io/projected/3c7ed38f-86c4-4046-8f3e-516265372dcb-kube-api-access-n8vh8\") pod \"cluster-monitoring-operator-76cf8966f8-tst5h\" (UID: \"3c7ed38f-86c4-4046-8f3e-516265372dcb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" Jul 22 02:58:44.821117 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.821255 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6-config\") pod \"console-operator-565b7b569d-94dxd\" (UID: \"fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6\") " pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:58:44.821117 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.821331 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6-serving-cert\") pod \"console-operator-565b7b569d-94dxd\" (UID: \"fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6\") " pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:58:44.821842 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:44.821436 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:58:44.821842 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:44.821490 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls podName:3c7ed38f-86c4-4046-8f3e-516265372dcb nodeName:}" failed. No retries permitted until 2026-07-22 02:58:45.321475374 +0000 UTC m=+54.512669655 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tst5h" (UID: "3c7ed38f-86c4-4046-8f3e-516265372dcb") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:58:44.822129 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.822101 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/3c7ed38f-86c4-4046-8f3e-516265372dcb-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-tst5h\" (UID: \"3c7ed38f-86c4-4046-8f3e-516265372dcb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" Jul 22 02:58:44.822284 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.822266 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6-config\") pod \"console-operator-565b7b569d-94dxd\" (UID: \"fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6\") " pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:58:44.822284 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.822277 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6-trusted-ca\") pod \"console-operator-565b7b569d-94dxd\" (UID: \"fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6\") " pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:58:44.824287 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.824255 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6-serving-cert\") pod \"console-operator-565b7b569d-94dxd\" (UID: \"fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6\") " pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:58:44.830930 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.830901 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6bgq\" (UniqueName: \"kubernetes.io/projected/fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6-kube-api-access-z6bgq\") pod \"console-operator-565b7b569d-94dxd\" (UID: \"fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6\") " pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:58:44.831477 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.831398 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8vh8\" (UniqueName: \"kubernetes.io/projected/3c7ed38f-86c4-4046-8f3e-516265372dcb-kube-api-access-n8vh8\") pod \"cluster-monitoring-operator-76cf8966f8-tst5h\" (UID: \"3c7ed38f-86c4-4046-8f3e-516265372dcb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" Jul 22 02:58:44.874397 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.874367 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-frhxj"] Jul 22 02:58:44.877229 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:58:44.877201 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74a3c85e_05c0_4ad6_addb_c70c0996ea10.slice/crio-f8219e60c1d908ac5f5a9842770e3316b07f125fdc91e4b9b317eb7fb3b509c6 WatchSource:0}: Error finding container f8219e60c1d908ac5f5a9842770e3316b07f125fdc91e4b9b317eb7fb3b509c6: Status 404 returned error can't find the container with id f8219e60c1d908ac5f5a9842770e3316b07f125fdc91e4b9b317eb7fb3b509c6 Jul 22 02:58:44.883655 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:44.883632 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:58:45.026953 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:45.026924 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-94dxd"] Jul 22 02:58:45.030175 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:58:45.030151 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdb88bbb_3960_48f5_b481_9a8c5a0c5fa6.slice/crio-f0fe55325304fb710f65e4acb4c44c077636fd74ff0fc3e8caf15b0bcea2fc01 WatchSource:0}: Error finding container f0fe55325304fb710f65e4acb4c44c077636fd74ff0fc3e8caf15b0bcea2fc01: Status 404 returned error can't find the container with id f0fe55325304fb710f65e4acb4c44c077636fd74ff0fc3e8caf15b0bcea2fc01 Jul 22 02:58:45.324584 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:45.324510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tst5h\" (UID: \"3c7ed38f-86c4-4046-8f3e-516265372dcb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" Jul 22 02:58:45.324705 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:45.324649 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:58:45.324705 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:45.324700 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls podName:3c7ed38f-86c4-4046-8f3e-516265372dcb nodeName:}" failed. No retries permitted until 2026-07-22 02:58:46.324687173 +0000 UTC m=+55.515881457 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tst5h" (UID: "3c7ed38f-86c4-4046-8f3e-516265372dcb") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:58:45.686576 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:45.686409 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-frhxj" event={"ID":"74a3c85e-05c0-4ad6-addb-c70c0996ea10","Type":"ContainerStarted","Data":"f8219e60c1d908ac5f5a9842770e3316b07f125fdc91e4b9b317eb7fb3b509c6"} Jul 22 02:58:45.687854 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:45.687730 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-94dxd" event={"ID":"fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6","Type":"ContainerStarted","Data":"f0fe55325304fb710f65e4acb4c44c077636fd74ff0fc3e8caf15b0bcea2fc01"} Jul 22 02:58:46.331799 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:46.331761 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tst5h\" (UID: \"3c7ed38f-86c4-4046-8f3e-516265372dcb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" Jul 22 02:58:46.332027 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:46.331907 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:58:46.332027 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:46.331981 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls podName:3c7ed38f-86c4-4046-8f3e-516265372dcb nodeName:}" failed. No retries permitted until 2026-07-22 02:58:48.331961269 +0000 UTC m=+57.523155553 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tst5h" (UID: "3c7ed38f-86c4-4046-8f3e-516265372dcb") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:58:46.692481 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:46.692439 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-frhxj" event={"ID":"74a3c85e-05c0-4ad6-addb-c70c0996ea10","Type":"ContainerStarted","Data":"3b9656269ea7f0fc9e57d2b971a06a980f7d0b6f88ea322daf6942890a33a8a5"} Jul 22 02:58:46.718559 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:46.718502 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-frhxj" podStartSLOduration=1.499028585 podStartE2EDuration="2.718482873s" totalImagesPullingTime="1.219454288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:58:46.717115297 +0000 UTC m=+55.908309599" watchObservedRunningTime="2026-07-22 02:58:46.718482873 +0000 UTC m=+55.909677178" Jul 22 02:58:47.696656 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:47.696628 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/0.log" Jul 22 02:58:47.697095 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:47.696666 2580 generic.go:356] "Generic (PLEG): container finished" podID="fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6" containerID="f2a8363c66b36aa4e6b0f36ccccf3388bcc7b0a92a1c80db7df26826030ffdf5" exitCode=255 Jul 22 02:58:47.697095 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:47.696744 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-94dxd" event={"ID":"fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6","Type":"ContainerDied","Data":"f2a8363c66b36aa4e6b0f36ccccf3388bcc7b0a92a1c80db7df26826030ffdf5"} Jul 22 02:58:47.697095 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:47.696936 2580 scope.go:117] "RemoveContainer" containerID="f2a8363c66b36aa4e6b0f36ccccf3388bcc7b0a92a1c80db7df26826030ffdf5" Jul 22 02:58:48.346113 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:48.346073 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tst5h\" (UID: \"3c7ed38f-86c4-4046-8f3e-516265372dcb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" Jul 22 02:58:48.346300 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:48.346196 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:58:48.346300 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:48.346250 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls podName:3c7ed38f-86c4-4046-8f3e-516265372dcb nodeName:}" failed. No retries permitted until 2026-07-22 02:58:52.346236117 +0000 UTC m=+61.537430401 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tst5h" (UID: "3c7ed38f-86c4-4046-8f3e-516265372dcb") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:58:48.700604 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:48.700579 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 02:58:48.701036 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:48.701019 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/0.log" Jul 22 02:58:48.701167 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:48.701062 2580 generic.go:356] "Generic (PLEG): container finished" podID="fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6" containerID="5ca27a645637a01911fa908ef29409445f7758f93923a0df5018bfa8ae900514" exitCode=255 Jul 22 02:58:48.701167 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:48.701113 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-94dxd" event={"ID":"fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6","Type":"ContainerDied","Data":"5ca27a645637a01911fa908ef29409445f7758f93923a0df5018bfa8ae900514"} Jul 22 02:58:48.701167 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:48.701166 2580 scope.go:117] "RemoveContainer" containerID="f2a8363c66b36aa4e6b0f36ccccf3388bcc7b0a92a1c80db7df26826030ffdf5" Jul 22 02:58:48.701313 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:48.701303 2580 scope.go:117] "RemoveContainer" containerID="5ca27a645637a01911fa908ef29409445f7758f93923a0df5018bfa8ae900514" Jul 22 02:58:48.701470 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:48.701451 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-94dxd_openshift-console-operator(fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6)\"" pod="openshift-console-operator/console-operator-565b7b569d-94dxd" podUID="fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6" Jul 22 02:58:49.705188 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:49.705162 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 02:58:49.705505 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:49.705492 2580 scope.go:117] "RemoveContainer" containerID="5ca27a645637a01911fa908ef29409445f7758f93923a0df5018bfa8ae900514" Jul 22 02:58:49.705683 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:49.705662 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-94dxd_openshift-console-operator(fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6)\"" pod="openshift-console-operator/console-operator-565b7b569d-94dxd" podUID="fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6" Jul 22 02:58:50.225362 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.225332 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pgnvv"] Jul 22 02:58:50.229573 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.229555 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:50.232037 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.232016 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 02:58:50.232037 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.232028 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 02:58:50.232869 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.232854 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zh2d9\"" Jul 22 02:58:50.232939 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.232888 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 02:58:50.233094 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.233081 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 02:58:50.245085 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.245063 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pgnvv"] Jul 22 02:58:50.359599 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.359569 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:50.359717 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.359617 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:50.359717 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.359650 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7018409a-3816-4591-a676-eb0e71ffed9d-data-volume\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:50.359825 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.359716 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z74t\" (UniqueName: \"kubernetes.io/projected/7018409a-3816-4591-a676-eb0e71ffed9d-kube-api-access-5z74t\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:50.359825 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.359752 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7018409a-3816-4591-a676-eb0e71ffed9d-crio-socket\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:50.460918 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.460886 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z74t\" (UniqueName: \"kubernetes.io/projected/7018409a-3816-4591-a676-eb0e71ffed9d-kube-api-access-5z74t\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:50.460992 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.460930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7018409a-3816-4591-a676-eb0e71ffed9d-crio-socket\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:50.461027 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.460995 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:50.461062 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.461031 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:50.461106 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.461090 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7018409a-3816-4591-a676-eb0e71ffed9d-crio-socket\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:50.461140 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:50.461098 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:58:50.461171 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.461138 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7018409a-3816-4591-a676-eb0e71ffed9d-data-volume\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:50.461171 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:50.461112 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:58:50.461232 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:50.461183 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm podName:7018409a-3816-4591-a676-eb0e71ffed9d nodeName:}" failed. No retries permitted until 2026-07-22 02:58:50.961163071 +0000 UTC m=+60.152357368 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgnvv" (UID: "7018409a-3816-4591-a676-eb0e71ffed9d") : configmap "kube-rbac-proxy" not found Jul 22 02:58:50.461232 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:50.461213 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls podName:7018409a-3816-4591-a676-eb0e71ffed9d nodeName:}" failed. No retries permitted until 2026-07-22 02:58:50.961203223 +0000 UTC m=+60.152397511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgnvv" (UID: "7018409a-3816-4591-a676-eb0e71ffed9d") : secret "insights-runtime-extractor-tls" not found Jul 22 02:58:50.461453 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.461436 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7018409a-3816-4591-a676-eb0e71ffed9d-data-volume\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:50.469410 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.469386 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z74t\" (UniqueName: \"kubernetes.io/projected/7018409a-3816-4591-a676-eb0e71ffed9d-kube-api-access-5z74t\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:50.965132 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.965088 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:50.965520 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:50.965199 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:50.965520 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:50.965207 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:58:50.965520 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:50.965267 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm podName:7018409a-3816-4591-a676-eb0e71ffed9d nodeName:}" failed. No retries permitted until 2026-07-22 02:58:51.965251336 +0000 UTC m=+61.156445617 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgnvv" (UID: "7018409a-3816-4591-a676-eb0e71ffed9d") : configmap "kube-rbac-proxy" not found Jul 22 02:58:50.965520 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:50.965289 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:58:50.965520 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:50.965335 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls podName:7018409a-3816-4591-a676-eb0e71ffed9d nodeName:}" failed. No retries permitted until 2026-07-22 02:58:51.965323641 +0000 UTC m=+61.156517921 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgnvv" (UID: "7018409a-3816-4591-a676-eb0e71ffed9d") : secret "insights-runtime-extractor-tls" not found Jul 22 02:58:51.973371 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:51.973340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:51.973371 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:51.973382 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:51.973771 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:51.973487 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:58:51.973771 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:51.973486 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:58:51.973771 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:51.973567 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm podName:7018409a-3816-4591-a676-eb0e71ffed9d nodeName:}" failed. No retries permitted until 2026-07-22 02:58:53.973553147 +0000 UTC m=+63.164747428 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgnvv" (UID: "7018409a-3816-4591-a676-eb0e71ffed9d") : configmap "kube-rbac-proxy" not found Jul 22 02:58:51.973771 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:51.973627 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls podName:7018409a-3816-4591-a676-eb0e71ffed9d nodeName:}" failed. No retries permitted until 2026-07-22 02:58:53.973613202 +0000 UTC m=+63.164807482 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgnvv" (UID: "7018409a-3816-4591-a676-eb0e71ffed9d") : secret "insights-runtime-extractor-tls" not found Jul 22 02:58:52.376060 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:52.376034 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tst5h\" (UID: \"3c7ed38f-86c4-4046-8f3e-516265372dcb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" Jul 22 02:58:52.376220 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:52.376145 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:58:52.376220 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:52.376191 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls podName:3c7ed38f-86c4-4046-8f3e-516265372dcb nodeName:}" failed. No retries permitted until 2026-07-22 02:59:00.376178264 +0000 UTC m=+69.567372545 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tst5h" (UID: "3c7ed38f-86c4-4046-8f3e-516265372dcb") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:58:52.629054 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:52.628989 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zptvl" Jul 22 02:58:52.879845 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:52.879749 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-rcf6g"] Jul 22 02:58:52.882210 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:52.882193 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-rcf6g" Jul 22 02:58:52.884606 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:52.884583 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-8rhbw\"" Jul 22 02:58:52.884606 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:52.884600 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 22 02:58:52.884764 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:52.884601 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 22 02:58:52.884764 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:52.884583 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 22 02:58:52.884764 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:52.884605 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 22 02:58:52.892030 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:52.892007 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-rcf6g"] Jul 22 02:58:52.980069 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:52.980041 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwhkw\" (UniqueName: \"kubernetes.io/projected/82db5f03-09a3-4185-a075-d8edf0399b05-kube-api-access-rwhkw\") pod \"service-ca-58598695f7-rcf6g\" (UID: \"82db5f03-09a3-4185-a075-d8edf0399b05\") " pod="openshift-service-ca/service-ca-58598695f7-rcf6g" Jul 22 02:58:52.980406 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:52.980086 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/82db5f03-09a3-4185-a075-d8edf0399b05-signing-key\") pod \"service-ca-58598695f7-rcf6g\" (UID: \"82db5f03-09a3-4185-a075-d8edf0399b05\") " pod="openshift-service-ca/service-ca-58598695f7-rcf6g" Jul 22 02:58:52.980406 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:52.980168 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/82db5f03-09a3-4185-a075-d8edf0399b05-signing-cabundle\") pod \"service-ca-58598695f7-rcf6g\" (UID: \"82db5f03-09a3-4185-a075-d8edf0399b05\") " pod="openshift-service-ca/service-ca-58598695f7-rcf6g" Jul 22 02:58:53.025482 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:53.025464 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7t98c_308ab8f1-8fb2-4833-b583-b22cf8e3cddd/dns-node-resolver/0.log" Jul 22 02:58:53.081102 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:53.081079 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/82db5f03-09a3-4185-a075-d8edf0399b05-signing-cabundle\") pod \"service-ca-58598695f7-rcf6g\" (UID: \"82db5f03-09a3-4185-a075-d8edf0399b05\") " pod="openshift-service-ca/service-ca-58598695f7-rcf6g" Jul 22 02:58:53.081206 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:53.081190 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwhkw\" (UniqueName: \"kubernetes.io/projected/82db5f03-09a3-4185-a075-d8edf0399b05-kube-api-access-rwhkw\") pod \"service-ca-58598695f7-rcf6g\" (UID: \"82db5f03-09a3-4185-a075-d8edf0399b05\") " pod="openshift-service-ca/service-ca-58598695f7-rcf6g" Jul 22 02:58:53.081261 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:53.081237 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/82db5f03-09a3-4185-a075-d8edf0399b05-signing-key\") pod \"service-ca-58598695f7-rcf6g\" (UID: \"82db5f03-09a3-4185-a075-d8edf0399b05\") " pod="openshift-service-ca/service-ca-58598695f7-rcf6g" Jul 22 02:58:53.081663 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:53.081644 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/82db5f03-09a3-4185-a075-d8edf0399b05-signing-cabundle\") pod \"service-ca-58598695f7-rcf6g\" (UID: \"82db5f03-09a3-4185-a075-d8edf0399b05\") " pod="openshift-service-ca/service-ca-58598695f7-rcf6g" Jul 22 02:58:53.083447 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:53.083428 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/82db5f03-09a3-4185-a075-d8edf0399b05-signing-key\") pod \"service-ca-58598695f7-rcf6g\" (UID: \"82db5f03-09a3-4185-a075-d8edf0399b05\") " pod="openshift-service-ca/service-ca-58598695f7-rcf6g" Jul 22 02:58:53.089020 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:53.088998 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwhkw\" (UniqueName: \"kubernetes.io/projected/82db5f03-09a3-4185-a075-d8edf0399b05-kube-api-access-rwhkw\") pod \"service-ca-58598695f7-rcf6g\" (UID: \"82db5f03-09a3-4185-a075-d8edf0399b05\") " pod="openshift-service-ca/service-ca-58598695f7-rcf6g" Jul 22 02:58:53.190788 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:53.190743 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-rcf6g" Jul 22 02:58:53.307078 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:53.307047 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-rcf6g"] Jul 22 02:58:53.310601 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:58:53.310565 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82db5f03_09a3_4185_a075_d8edf0399b05.slice/crio-6c431e4a9384865dbe0094fe2de2bd1ee6b3f07619ef0a8e4f96f1aa751e8079 WatchSource:0}: Error finding container 6c431e4a9384865dbe0094fe2de2bd1ee6b3f07619ef0a8e4f96f1aa751e8079: Status 404 returned error can't find the container with id 6c431e4a9384865dbe0094fe2de2bd1ee6b3f07619ef0a8e4f96f1aa751e8079 Jul 22 02:58:53.715473 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:53.715433 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-rcf6g" event={"ID":"82db5f03-09a3-4185-a075-d8edf0399b05","Type":"ContainerStarted","Data":"6c431e4a9384865dbe0094fe2de2bd1ee6b3f07619ef0a8e4f96f1aa751e8079"} Jul 22 02:58:53.822594 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:53.822563 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-99vc7_91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b/node-ca/0.log" Jul 22 02:58:53.989266 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:53.989182 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:53.989266 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:53.989232 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:53.989740 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:53.989347 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:58:53.989740 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:53.989418 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls podName:7018409a-3816-4591-a676-eb0e71ffed9d nodeName:}" failed. No retries permitted until 2026-07-22 02:58:57.989395906 +0000 UTC m=+67.180590210 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgnvv" (UID: "7018409a-3816-4591-a676-eb0e71ffed9d") : secret "insights-runtime-extractor-tls" not found Jul 22 02:58:53.989740 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:53.989353 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:58:53.989740 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:53.989471 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm podName:7018409a-3816-4591-a676-eb0e71ffed9d nodeName:}" failed. No retries permitted until 2026-07-22 02:58:57.989460357 +0000 UTC m=+67.180654644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgnvv" (UID: "7018409a-3816-4591-a676-eb0e71ffed9d") : configmap "kube-rbac-proxy" not found Jul 22 02:58:54.822803 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:54.822768 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r6wl8_7981d361-b9b8-4e56-97ad-928079266fb8/serve-healthcheck-canary/0.log" Jul 22 02:58:54.884038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:54.883993 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:58:54.884225 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:54.884109 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:58:54.884547 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:54.884526 2580 scope.go:117] "RemoveContainer" containerID="5ca27a645637a01911fa908ef29409445f7758f93923a0df5018bfa8ae900514" Jul 22 02:58:54.884763 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:54.884737 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-94dxd_openshift-console-operator(fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6)\"" pod="openshift-console-operator/console-operator-565b7b569d-94dxd" podUID="fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6" Jul 22 02:58:55.723196 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:55.723160 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-rcf6g" event={"ID":"82db5f03-09a3-4185-a075-d8edf0399b05","Type":"ContainerStarted","Data":"25726a6900104f0aa57c84ec570bf03ea553881179544eb2449580810d45bb64"} Jul 22 02:58:55.740290 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:55.740245 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-rcf6g" podStartSLOduration=2.053857934 podStartE2EDuration="3.740230866s" totalImagesPullingTime="1.686372932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:58:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:58:55.739547851 +0000 UTC m=+64.930742154" watchObservedRunningTime="2026-07-22 02:58:55.740230866 +0000 UTC m=+64.931425372" Jul 22 02:58:57.014438 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:57.014403 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tw9c\" (UID: \"183fb194-f313-4528-bf1b-c457f3174d9d\") " pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:57.014950 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:57.014479 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs\") pod \"network-metrics-daemon-mlxzc\" (UID: \"31d06428-0968-400f-8b93-86bacf6ec123\") " pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 02:58:57.017617 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:57.017595 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 02:58:57.017723 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:57.017645 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 02:58:57.025311 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:57.025291 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 22 02:58:57.025410 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:57.025354 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs podName:31d06428-0968-400f-8b93-86bacf6ec123 nodeName:}" failed. No retries permitted until 2026-07-22 03:00:01.025334323 +0000 UTC m=+130.216528603 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs") pod "network-metrics-daemon-mlxzc" (UID: "31d06428-0968-400f-8b93-86bacf6ec123") : secret "metrics-daemon-secret" not found Jul 22 02:58:57.027461 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:57.027442 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/183fb194-f313-4528-bf1b-c457f3174d9d-original-pull-secret\") pod \"global-pull-secret-syncer-9tw9c\" (UID: \"183fb194-f313-4528-bf1b-c457f3174d9d\") " pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:57.084302 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:57.084276 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tw9c" Jul 22 02:58:57.115611 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:57.115584 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xwcm\" (UniqueName: \"kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm\") pod \"network-check-target-szjw4\" (UID: \"5588cb65-fb3d-4955-98d4-5f5d5ad5234c\") " pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:57.118374 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:57.118352 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 02:58:57.129498 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:57.129476 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 02:58:57.139086 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:57.139057 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xwcm\" (UniqueName: \"kubernetes.io/projected/5588cb65-fb3d-4955-98d4-5f5d5ad5234c-kube-api-access-5xwcm\") pod \"network-check-target-szjw4\" (UID: \"5588cb65-fb3d-4955-98d4-5f5d5ad5234c\") " pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:57.206531 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:57.206505 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9tw9c"] Jul 22 02:58:57.209201 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:58:57.209176 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod183fb194_f313_4528_bf1b_c457f3174d9d.slice/crio-99def368f6de7cd95a39540a1f27c800842e99ec20caf551d17797714af81471 WatchSource:0}: Error finding container 99def368f6de7cd95a39540a1f27c800842e99ec20caf551d17797714af81471: Status 404 returned error can't find the container with id 99def368f6de7cd95a39540a1f27c800842e99ec20caf551d17797714af81471 Jul 22 02:58:57.381865 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:57.381844 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nztgp\"" Jul 22 02:58:57.390070 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:57.390045 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:58:57.515917 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:57.515842 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-szjw4"] Jul 22 02:58:57.519218 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:58:57.519193 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5588cb65_fb3d_4955_98d4_5f5d5ad5234c.slice/crio-58c4a67b38e6f4f6b9be63684a6f2d08fb4c6ea19ea8a620fe8063fd9e9f8484 WatchSource:0}: Error finding container 58c4a67b38e6f4f6b9be63684a6f2d08fb4c6ea19ea8a620fe8063fd9e9f8484: Status 404 returned error can't find the container with id 58c4a67b38e6f4f6b9be63684a6f2d08fb4c6ea19ea8a620fe8063fd9e9f8484 Jul 22 02:58:57.730698 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:57.730614 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-szjw4" event={"ID":"5588cb65-fb3d-4955-98d4-5f5d5ad5234c","Type":"ContainerStarted","Data":"58c4a67b38e6f4f6b9be63684a6f2d08fb4c6ea19ea8a620fe8063fd9e9f8484"} Jul 22 02:58:57.731968 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:57.731937 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9tw9c" event={"ID":"183fb194-f313-4528-bf1b-c457f3174d9d","Type":"ContainerStarted","Data":"99def368f6de7cd95a39540a1f27c800842e99ec20caf551d17797714af81471"} Jul 22 02:58:58.023917 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:58.023818 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:58.023917 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:58.023898 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:58:58.024405 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:58.023960 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:58:58.024405 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:58.024034 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls podName:7018409a-3816-4591-a676-eb0e71ffed9d nodeName:}" failed. No retries permitted until 2026-07-22 02:59:06.024012917 +0000 UTC m=+75.215207206 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgnvv" (UID: "7018409a-3816-4591-a676-eb0e71ffed9d") : secret "insights-runtime-extractor-tls" not found Jul 22 02:58:58.024405 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:58.024035 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:58:58.024405 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:58.024099 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm podName:7018409a-3816-4591-a676-eb0e71ffed9d nodeName:}" failed. No retries permitted until 2026-07-22 02:59:06.024079963 +0000 UTC m=+75.215274248 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgnvv" (UID: "7018409a-3816-4591-a676-eb0e71ffed9d") : configmap "kube-rbac-proxy" not found Jul 22 02:58:59.133525 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:59.133435 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:59.133525 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:59.133519 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:58:59.134066 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:59.133697 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:58:59.134066 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:59.133718 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-984955c4f-ff8rk: secret "image-registry-tls" not found Jul 22 02:58:59.134066 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:58:59.133776 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls podName:01f1c1c1-3d02-4a21-8468-4c8c30b91c5a nodeName:}" failed. No retries permitted until 2026-07-22 02:59:31.133756979 +0000 UTC m=+100.324951275 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls") pod "image-registry-984955c4f-ff8rk" (UID: "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a") : secret "image-registry-tls" not found Jul 22 02:58:59.139197 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:59.139172 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/33beac89-d63c-4c71-82db-1790e90a1447-metrics-tls\") pod \"dns-default-b46cz\" (UID: \"33beac89-d63c-4c71-82db-1790e90a1447\") " pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:59.355260 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:59.355132 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-sn76t\"" Jul 22 02:58:59.363482 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:59.363448 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b46cz" Jul 22 02:58:59.534280 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:59.533031 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b46cz"] Jul 22 02:58:59.537854 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:58:59.537798 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33beac89_d63c_4c71_82db_1790e90a1447.slice/crio-bd60fff5a26cff10462c226f7d73303e8e4f79d22d540db35a7b0231e76c341c WatchSource:0}: Error finding container bd60fff5a26cff10462c226f7d73303e8e4f79d22d540db35a7b0231e76c341c: Status 404 returned error can't find the container with id bd60fff5a26cff10462c226f7d73303e8e4f79d22d540db35a7b0231e76c341c Jul 22 02:58:59.739717 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:58:59.739634 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b46cz" event={"ID":"33beac89-d63c-4c71-82db-1790e90a1447","Type":"ContainerStarted","Data":"bd60fff5a26cff10462c226f7d73303e8e4f79d22d540db35a7b0231e76c341c"} Jul 22 02:59:00.445453 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:00.445410 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tst5h\" (UID: \"3c7ed38f-86c4-4046-8f3e-516265372dcb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" Jul 22 02:59:00.445884 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:59:00.445591 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:59:00.445884 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:59:00.445664 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls podName:3c7ed38f-86c4-4046-8f3e-516265372dcb nodeName:}" failed. No retries permitted until 2026-07-22 02:59:16.445643497 +0000 UTC m=+85.636837794 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-tst5h" (UID: "3c7ed38f-86c4-4046-8f3e-516265372dcb") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:59:03.758253 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:03.758209 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-szjw4" event={"ID":"5588cb65-fb3d-4955-98d4-5f5d5ad5234c","Type":"ContainerStarted","Data":"1b9d07c23341f1ba96022b760cde4dec42714159c12ff72e44be837dd1ae5100"} Jul 22 02:59:03.758673 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:03.758314 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:59:03.759705 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:03.759676 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9tw9c" event={"ID":"183fb194-f313-4528-bf1b-c457f3174d9d","Type":"ContainerStarted","Data":"2ca38e9c27e7fb4e1e1d27c3e4affb29105e809d0c8c199c7e2ec7d6ed39a324"} Jul 22 02:59:03.761325 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:03.761304 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b46cz" event={"ID":"33beac89-d63c-4c71-82db-1790e90a1447","Type":"ContainerStarted","Data":"9313d55f58d3a6f744d42b119203dac70d86bf5417c6f68e3f5ee5bce721f24c"} Jul 22 02:59:03.761404 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:03.761343 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b46cz" event={"ID":"33beac89-d63c-4c71-82db-1790e90a1447","Type":"ContainerStarted","Data":"5271e05c5080bdd0b6c9ed114f2b3f1a81cbc9c9a61040e63ac385f85377e5a5"} Jul 22 02:59:03.761467 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:03.761434 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b46cz" Jul 22 02:59:03.777291 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:03.777254 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-szjw4" podStartSLOduration=67.60186351 podStartE2EDuration="1m12.777242853s" totalImagesPullingTime="5.175379343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:59:03.77708966 +0000 UTC m=+72.968283962" watchObservedRunningTime="2026-07-22 02:59:03.777242853 +0000 UTC m=+72.968437191" Jul 22 02:59:03.791892 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:03.791856 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9tw9c" podStartSLOduration=67.294462108 podStartE2EDuration="1m12.791825771s" totalImagesPullingTime="5.497363663s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:59:03.791248974 +0000 UTC m=+72.982443276" watchObservedRunningTime="2026-07-22 02:59:03.791825771 +0000 UTC m=+72.983020073" Jul 22 02:59:03.808287 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:03.808248 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b46cz" podStartSLOduration=33.604082304 podStartE2EDuration="36.808234828s" totalImagesPullingTime="3.204152524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:58:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:59:03.807118629 +0000 UTC m=+72.998312949" watchObservedRunningTime="2026-07-22 02:59:03.808234828 +0000 UTC m=+72.999429130" Jul 22 02:59:06.092093 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:06.092057 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:59:06.092093 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:06.092095 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:59:06.092582 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:06.092548 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7018409a-3816-4591-a676-eb0e71ffed9d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:59:06.094337 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:06.094312 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7018409a-3816-4591-a676-eb0e71ffed9d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgnvv\" (UID: \"7018409a-3816-4591-a676-eb0e71ffed9d\") " pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:59:06.141408 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:06.141382 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zh2d9\"" Jul 22 02:59:06.149610 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:06.149591 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pgnvv" Jul 22 02:59:06.281192 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:06.281164 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pgnvv"] Jul 22 02:59:06.284568 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:59:06.284543 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7018409a_3816_4591_a676_eb0e71ffed9d.slice/crio-76ced9ed9078b172282e894bf4f44c5cf8bc5e1fff111c0667090b4352a9144c WatchSource:0}: Error finding container 76ced9ed9078b172282e894bf4f44c5cf8bc5e1fff111c0667090b4352a9144c: Status 404 returned error can't find the container with id 76ced9ed9078b172282e894bf4f44c5cf8bc5e1fff111c0667090b4352a9144c Jul 22 02:59:06.773879 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:06.773841 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pgnvv" event={"ID":"7018409a-3816-4591-a676-eb0e71ffed9d","Type":"ContainerStarted","Data":"76ced9ed9078b172282e894bf4f44c5cf8bc5e1fff111c0667090b4352a9144c"} Jul 22 02:59:07.361755 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:07.361723 2580 scope.go:117] "RemoveContainer" containerID="5ca27a645637a01911fa908ef29409445f7758f93923a0df5018bfa8ae900514" Jul 22 02:59:07.779371 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:07.779297 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 02:59:07.779371 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:07.779360 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-94dxd" event={"ID":"fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6","Type":"ContainerStarted","Data":"4cec61707ed1d7d5f294bef955bbeafcad8e7f5e27428a55da50f7b7e61438de"} Jul 22 02:59:07.779777 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:07.779750 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:59:07.797065 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:07.797024 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-94dxd" podStartSLOduration=21.73794964 podStartE2EDuration="23.797009545s" totalImagesPullingTime="2.059059905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:59:07.795719987 +0000 UTC m=+76.986914311" watchObservedRunningTime="2026-07-22 02:59:07.797009545 +0000 UTC m=+76.988203847" Jul 22 02:59:07.952782 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:07.952751 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-94dxd" Jul 22 02:59:11.796190 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:11.796147 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pgnvv" event={"ID":"7018409a-3816-4591-a676-eb0e71ffed9d","Type":"ContainerStarted","Data":"51a909f7e53ae827ace60191ab17736e503751a20c28cde60346cc92b1e03ecb"} Jul 22 02:59:13.767169 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:13.767048 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b46cz" Jul 22 02:59:14.806800 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:14.806720 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pgnvv" event={"ID":"7018409a-3816-4591-a676-eb0e71ffed9d","Type":"ContainerStarted","Data":"16e82e40e91c581415975bccaa73d3e13822472a4c7673d64ce83934f422ba02"} Jul 22 02:59:16.474679 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:16.474579 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tst5h\" (UID: \"3c7ed38f-86c4-4046-8f3e-516265372dcb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" Jul 22 02:59:16.477129 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:16.477103 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/3c7ed38f-86c4-4046-8f3e-516265372dcb-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-tst5h\" (UID: \"3c7ed38f-86c4-4046-8f3e-516265372dcb\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" Jul 22 02:59:16.661824 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:16.661788 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9b764\"" Jul 22 02:59:16.669280 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:16.669254 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" Jul 22 02:59:18.181632 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.181607 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h"] Jul 22 02:59:18.184588 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:59:18.184563 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c7ed38f_86c4_4046_8f3e_516265372dcb.slice/crio-3ae94aa87c8a7d955d451cfd0e2f1f32f2543665498046298e096b66333d8aa7 WatchSource:0}: Error finding container 3ae94aa87c8a7d955d451cfd0e2f1f32f2543665498046298e096b66333d8aa7: Status 404 returned error can't find the container with id 3ae94aa87c8a7d955d451cfd0e2f1f32f2543665498046298e096b66333d8aa7 Jul 22 02:59:18.450090 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.450057 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7"] Jul 22 02:59:18.453056 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.453039 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7" Jul 22 02:59:18.455334 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.455312 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 22 02:59:18.456216 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.456200 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 22 02:59:18.456288 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.456200 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 22 02:59:18.456288 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.456238 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 22 02:59:18.468275 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.468244 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7"] Jul 22 02:59:18.577091 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.577068 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-697458475c-6xmhl"] Jul 22 02:59:18.580010 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.579992 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.583631 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.583609 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 22 02:59:18.583869 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.583856 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-srtb2\"" Jul 22 02:59:18.584121 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.584105 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 02:59:18.584200 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.584105 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 22 02:59:18.584200 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.584163 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 02:59:18.584302 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.584205 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 02:59:18.584302 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.584232 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 02:59:18.584412 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.584398 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 02:59:18.591319 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.591300 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a3cbc76-1fb8-4780-bb42-0dc1593f13bc-tmp\") pod \"klusterlet-addon-workmgr-64ddfd6d5c-md2v7\" (UID: \"4a3cbc76-1fb8-4780-bb42-0dc1593f13bc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7" Jul 22 02:59:18.591418 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.591324 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4a3cbc76-1fb8-4780-bb42-0dc1593f13bc-klusterlet-config\") pod \"klusterlet-addon-workmgr-64ddfd6d5c-md2v7\" (UID: \"4a3cbc76-1fb8-4780-bb42-0dc1593f13bc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7" Jul 22 02:59:18.591418 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.591375 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsmwc\" (UniqueName: \"kubernetes.io/projected/4a3cbc76-1fb8-4780-bb42-0dc1593f13bc-kube-api-access-wsmwc\") pod \"klusterlet-addon-workmgr-64ddfd6d5c-md2v7\" (UID: \"4a3cbc76-1fb8-4780-bb42-0dc1593f13bc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7" Jul 22 02:59:18.593092 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.593073 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-vd8sx"] Jul 22 02:59:18.595915 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.595898 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-vd8sx" Jul 22 02:59:18.598622 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.598607 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-fwwq4\"" Jul 22 02:59:18.602247 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.602227 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-697458475c-6xmhl"] Jul 22 02:59:18.612300 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.612281 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-vd8sx"] Jul 22 02:59:18.692447 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.692425 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncbjk\" (UniqueName: \"kubernetes.io/projected/8c542afe-969b-42c5-9dc5-dc4eb2586201-kube-api-access-ncbjk\") pod \"downloads-5bc67dcd46-vd8sx\" (UID: \"8c542afe-969b-42c5-9dc5-dc4eb2586201\") " pod="openshift-console/downloads-5bc67dcd46-vd8sx" Jul 22 02:59:18.692585 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.692461 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a3cbc76-1fb8-4780-bb42-0dc1593f13bc-tmp\") pod \"klusterlet-addon-workmgr-64ddfd6d5c-md2v7\" (UID: \"4a3cbc76-1fb8-4780-bb42-0dc1593f13bc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7" Jul 22 02:59:18.692585 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.692480 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4a3cbc76-1fb8-4780-bb42-0dc1593f13bc-klusterlet-config\") pod \"klusterlet-addon-workmgr-64ddfd6d5c-md2v7\" (UID: \"4a3cbc76-1fb8-4780-bb42-0dc1593f13bc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7" Jul 22 02:59:18.692585 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.692505 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-oauth-config\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.692585 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.692532 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-serving-cert\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.692585 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.692563 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-config\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.692863 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.692630 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-oauth-serving-cert\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.692863 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.692658 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsmwc\" (UniqueName: \"kubernetes.io/projected/4a3cbc76-1fb8-4780-bb42-0dc1593f13bc-kube-api-access-wsmwc\") pod \"klusterlet-addon-workmgr-64ddfd6d5c-md2v7\" (UID: \"4a3cbc76-1fb8-4780-bb42-0dc1593f13bc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7" Jul 22 02:59:18.692863 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.692685 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-service-ca\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.692863 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.692755 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjd5c\" (UniqueName: \"kubernetes.io/projected/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-kube-api-access-kjd5c\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.692863 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.692858 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4a3cbc76-1fb8-4780-bb42-0dc1593f13bc-tmp\") pod \"klusterlet-addon-workmgr-64ddfd6d5c-md2v7\" (UID: \"4a3cbc76-1fb8-4780-bb42-0dc1593f13bc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7" Jul 22 02:59:18.695275 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.695256 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4a3cbc76-1fb8-4780-bb42-0dc1593f13bc-klusterlet-config\") pod \"klusterlet-addon-workmgr-64ddfd6d5c-md2v7\" (UID: \"4a3cbc76-1fb8-4780-bb42-0dc1593f13bc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7" Jul 22 02:59:18.714777 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.714728 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsmwc\" (UniqueName: \"kubernetes.io/projected/4a3cbc76-1fb8-4780-bb42-0dc1593f13bc-kube-api-access-wsmwc\") pod \"klusterlet-addon-workmgr-64ddfd6d5c-md2v7\" (UID: \"4a3cbc76-1fb8-4780-bb42-0dc1593f13bc\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7" Jul 22 02:59:18.762206 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.762182 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7" Jul 22 02:59:18.793345 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.793314 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-oauth-config\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.793457 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.793356 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-serving-cert\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.793457 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.793395 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-config\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.793538 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.793482 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-oauth-serving-cert\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.793538 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.793517 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-service-ca\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.793635 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.793583 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjd5c\" (UniqueName: \"kubernetes.io/projected/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-kube-api-access-kjd5c\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.793635 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.793618 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncbjk\" (UniqueName: \"kubernetes.io/projected/8c542afe-969b-42c5-9dc5-dc4eb2586201-kube-api-access-ncbjk\") pod \"downloads-5bc67dcd46-vd8sx\" (UID: \"8c542afe-969b-42c5-9dc5-dc4eb2586201\") " pod="openshift-console/downloads-5bc67dcd46-vd8sx" Jul 22 02:59:18.794215 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.794188 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-config\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.794321 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.794264 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-oauth-serving-cert\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.794523 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.794500 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-service-ca\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.795936 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.795910 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-oauth-config\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.796101 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.796086 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-serving-cert\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.805917 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.805880 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncbjk\" (UniqueName: \"kubernetes.io/projected/8c542afe-969b-42c5-9dc5-dc4eb2586201-kube-api-access-ncbjk\") pod \"downloads-5bc67dcd46-vd8sx\" (UID: \"8c542afe-969b-42c5-9dc5-dc4eb2586201\") " pod="openshift-console/downloads-5bc67dcd46-vd8sx" Jul 22 02:59:18.806286 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.806073 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjd5c\" (UniqueName: \"kubernetes.io/projected/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-kube-api-access-kjd5c\") pod \"console-697458475c-6xmhl\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.830656 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.830519 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pgnvv" event={"ID":"7018409a-3816-4591-a676-eb0e71ffed9d","Type":"ContainerStarted","Data":"d9bba0359a565396577207e81e50094e56be39501a8a9957298e2b7f91482a72"} Jul 22 02:59:18.831892 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.831859 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" event={"ID":"3c7ed38f-86c4-4046-8f3e-516265372dcb","Type":"ContainerStarted","Data":"3ae94aa87c8a7d955d451cfd0e2f1f32f2543665498046298e096b66333d8aa7"} Jul 22 02:59:18.849307 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.849264 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pgnvv" podStartSLOduration=17.127456479 podStartE2EDuration="28.849249664s" totalImagesPullingTime="11.721793185s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:58:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:59:18.848851113 +0000 UTC m=+88.040045410" watchObservedRunningTime="2026-07-22 02:59:18.849249664 +0000 UTC m=+88.040443967" Jul 22 02:59:18.889192 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.889162 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:18.904672 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.904599 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7"] Jul 22 02:59:18.904816 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:18.904677 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-vd8sx" Jul 22 02:59:18.907909 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:59:18.907888 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a3cbc76_1fb8_4780_bb42_0dc1593f13bc.slice/crio-f00c1714d9bb4f1afffd76269a8a7f584aa29c15465cd20f588b4815ae242716 WatchSource:0}: Error finding container f00c1714d9bb4f1afffd76269a8a7f584aa29c15465cd20f588b4815ae242716: Status 404 returned error can't find the container with id f00c1714d9bb4f1afffd76269a8a7f584aa29c15465cd20f588b4815ae242716 Jul 22 02:59:19.040978 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:19.040946 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-697458475c-6xmhl"] Jul 22 02:59:19.043775 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:59:19.043743 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode75b4bc8_9ca1_4bbe_b91d_35b32dda2ad0.slice/crio-5b4057fbaec839fe134fe624d2363fd835770f0777cba09e9cf61838d61fae5a WatchSource:0}: Error finding container 5b4057fbaec839fe134fe624d2363fd835770f0777cba09e9cf61838d61fae5a: Status 404 returned error can't find the container with id 5b4057fbaec839fe134fe624d2363fd835770f0777cba09e9cf61838d61fae5a Jul 22 02:59:19.057518 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:19.057492 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-vd8sx"] Jul 22 02:59:19.060642 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:59:19.060617 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c542afe_969b_42c5_9dc5_dc4eb2586201.slice/crio-179d29e0b59651478ab47a025524ebf9e73f86db23db11b20633d7444da93618 WatchSource:0}: Error finding container 179d29e0b59651478ab47a025524ebf9e73f86db23db11b20633d7444da93618: Status 404 returned error can't find the container with id 179d29e0b59651478ab47a025524ebf9e73f86db23db11b20633d7444da93618 Jul 22 02:59:19.838456 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:19.838368 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697458475c-6xmhl" event={"ID":"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0","Type":"ContainerStarted","Data":"5b4057fbaec839fe134fe624d2363fd835770f0777cba09e9cf61838d61fae5a"} Jul 22 02:59:19.840038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:19.839961 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7" event={"ID":"4a3cbc76-1fb8-4780-bb42-0dc1593f13bc","Type":"ContainerStarted","Data":"f00c1714d9bb4f1afffd76269a8a7f584aa29c15465cd20f588b4815ae242716"} Jul 22 02:59:19.851046 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:19.845566 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-vd8sx" event={"ID":"8c542afe-969b-42c5-9dc5-dc4eb2586201","Type":"ContainerStarted","Data":"179d29e0b59651478ab47a025524ebf9e73f86db23db11b20633d7444da93618"} Jul 22 02:59:20.659900 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:20.659114 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ctjvc"] Jul 22 02:59:20.663249 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:20.662663 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ctjvc" Jul 22 02:59:20.665817 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:20.665506 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 22 02:59:20.667585 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:20.667560 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-vhs4d\"" Jul 22 02:59:20.675808 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:20.675757 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ctjvc"] Jul 22 02:59:20.815177 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:20.815067 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d4693632-3bd9-4778-b305-9ce70fe4b162-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-ctjvc\" (UID: \"d4693632-3bd9-4778-b305-9ce70fe4b162\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ctjvc" Jul 22 02:59:20.854080 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:20.853770 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" event={"ID":"3c7ed38f-86c4-4046-8f3e-516265372dcb","Type":"ContainerStarted","Data":"51036b22e4cf65884d2074d61d35d9c43a6f2462d85ab3cbf809759c96c8213a"} Jul 22 02:59:20.872578 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:20.872367 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-tst5h" podStartSLOduration=34.956923751 podStartE2EDuration="36.872348086s" totalImagesPullingTime="1.915424335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:59:20.872089725 +0000 UTC m=+90.063284029" watchObservedRunningTime="2026-07-22 02:59:20.872348086 +0000 UTC m=+90.063542391" Jul 22 02:59:20.916536 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:20.915961 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d4693632-3bd9-4778-b305-9ce70fe4b162-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-ctjvc\" (UID: \"d4693632-3bd9-4778-b305-9ce70fe4b162\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ctjvc" Jul 22 02:59:20.916536 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:59:20.916216 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 22 02:59:20.916536 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:59:20.916279 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4693632-3bd9-4778-b305-9ce70fe4b162-tls-certificates podName:d4693632-3bd9-4778-b305-9ce70fe4b162 nodeName:}" failed. No retries permitted until 2026-07-22 02:59:21.416259776 +0000 UTC m=+90.607454069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/d4693632-3bd9-4778-b305-9ce70fe4b162-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-ctjvc" (UID: "d4693632-3bd9-4778-b305-9ce70fe4b162") : secret "prometheus-operator-admission-webhook-tls" not found Jul 22 02:59:21.420134 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:21.420100 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d4693632-3bd9-4778-b305-9ce70fe4b162-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-ctjvc\" (UID: \"d4693632-3bd9-4778-b305-9ce70fe4b162\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ctjvc" Jul 22 02:59:21.423023 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:21.422993 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d4693632-3bd9-4778-b305-9ce70fe4b162-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-ctjvc\" (UID: \"d4693632-3bd9-4778-b305-9ce70fe4b162\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ctjvc" Jul 22 02:59:21.586850 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:21.586603 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ctjvc" Jul 22 02:59:23.669473 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:23.669443 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ctjvc"] Jul 22 02:59:23.674170 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:59:23.674140 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4693632_3bd9_4778_b305_9ce70fe4b162.slice/crio-41ac98b54710c512260130e3c06fa7d756dba767f66ccdd7fa43e51c0ce0a586 WatchSource:0}: Error finding container 41ac98b54710c512260130e3c06fa7d756dba767f66ccdd7fa43e51c0ce0a586: Status 404 returned error can't find the container with id 41ac98b54710c512260130e3c06fa7d756dba767f66ccdd7fa43e51c0ce0a586 Jul 22 02:59:23.870115 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:23.870064 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697458475c-6xmhl" event={"ID":"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0","Type":"ContainerStarted","Data":"eb9487d356d90b930c04f449735a915a7465876920e315fcac2b675c384a76d5"} Jul 22 02:59:23.871787 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:23.871757 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ctjvc" event={"ID":"d4693632-3bd9-4778-b305-9ce70fe4b162","Type":"ContainerStarted","Data":"41ac98b54710c512260130e3c06fa7d756dba767f66ccdd7fa43e51c0ce0a586"} Jul 22 02:59:23.893709 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:23.893645 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-697458475c-6xmhl" podStartSLOduration=1.363806407 podStartE2EDuration="5.893624307s" totalImagesPullingTime="4.5298179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:59:23.892572189 +0000 UTC m=+93.083766516" watchObservedRunningTime="2026-07-22 02:59:23.893624307 +0000 UTC m=+93.084818613" Jul 22 02:59:25.881526 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:25.881488 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7" event={"ID":"4a3cbc76-1fb8-4780-bb42-0dc1593f13bc","Type":"ContainerStarted","Data":"47e47d485b0406037b171d18f83629012bf45bdeb3d07bea1e9819340f376eab"} Jul 22 02:59:25.881989 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:25.881713 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7" Jul 22 02:59:25.883879 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:25.883855 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7" Jul 22 02:59:25.900106 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:25.900046 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-64ddfd6d5c-md2v7" podStartSLOduration=1.491703065 podStartE2EDuration="7.900029376s" totalImagesPullingTime="6.408326311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:59:25.899302173 +0000 UTC m=+95.090496475" watchObservedRunningTime="2026-07-22 02:59:25.900029376 +0000 UTC m=+95.091223680" Jul 22 02:59:26.888027 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:26.887976 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ctjvc" event={"ID":"d4693632-3bd9-4778-b305-9ce70fe4b162","Type":"ContainerStarted","Data":"db8bf0fc25a5c8e916faa9f9896f7919a0c374ced403e1450fa4dd95ee0da06a"} Jul 22 02:59:26.888615 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:26.888573 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ctjvc" Jul 22 02:59:26.893690 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:26.893610 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ctjvc" Jul 22 02:59:26.905109 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:26.905066 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-ctjvc" podStartSLOduration=4.494353449 podStartE2EDuration="6.90505187s" totalImagesPullingTime="2.410698421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:59:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:59:26.904767921 +0000 UTC m=+96.095962226" watchObservedRunningTime="2026-07-22 02:59:26.90505187 +0000 UTC m=+96.096246173" Jul 22 02:59:28.889967 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:28.889928 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:28.890436 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:28.890001 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:28.895803 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:28.895779 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:28.900085 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:28.900064 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-697458475c-6xmhl" Jul 22 02:59:31.207243 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:31.207151 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:59:31.209726 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:31.209702 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls\") pod \"image-registry-984955c4f-ff8rk\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:59:31.442119 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:31.442085 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7vzzs\"" Jul 22 02:59:31.446559 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:31.446534 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:59:33.105008 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.104975 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-4cf24"] Jul 22 02:59:33.195851 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.195690 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-f96v5"] Jul 22 02:59:33.196030 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.195884 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.200673 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.200610 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-bnd95\"" Jul 22 02:59:33.201041 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.200805 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 22 02:59:33.201041 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.200870 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 22 02:59:33.201163 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.201097 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 22 02:59:33.202280 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.201307 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 02:59:33.217859 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.217800 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-4cf24"] Jul 22 02:59:33.218038 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.217959 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.220434 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.220270 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 02:59:33.221399 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.220769 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 02:59:33.221399 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.221014 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ffkrv\"" Jul 22 02:59:33.221399 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.221237 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 02:59:33.324816 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.324782 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9668d2df-a646-45c2-b675-6b9a0b4586d5-root\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.325007 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.324846 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3158f020-696e-4d3f-992c-054efbaa67dc-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.325007 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.324878 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3158f020-696e-4d3f-992c-054efbaa67dc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.325007 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.324919 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3158f020-696e-4d3f-992c-054efbaa67dc-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.325007 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.324947 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl47p\" (UniqueName: \"kubernetes.io/projected/3158f020-696e-4d3f-992c-054efbaa67dc-kube-api-access-bl47p\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.325007 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.324987 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9668d2df-a646-45c2-b675-6b9a0b4586d5-node-exporter-tls\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.325254 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.325010 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9668d2df-a646-45c2-b675-6b9a0b4586d5-metrics-client-ca\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.325254 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.325038 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9668d2df-a646-45c2-b675-6b9a0b4586d5-sys\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.325254 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.325061 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9668d2df-a646-45c2-b675-6b9a0b4586d5-node-exporter-textfile\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.325254 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.325088 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3158f020-696e-4d3f-992c-054efbaa67dc-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.325254 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.325110 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v97zv\" (UniqueName: \"kubernetes.io/projected/9668d2df-a646-45c2-b675-6b9a0b4586d5-kube-api-access-v97zv\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.325254 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.325158 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9668d2df-a646-45c2-b675-6b9a0b4586d5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.325254 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.325183 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9668d2df-a646-45c2-b675-6b9a0b4586d5-node-exporter-accelerators-collector-config\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.325254 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.325212 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9668d2df-a646-45c2-b675-6b9a0b4586d5-node-exporter-wtmp\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.325254 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.325241 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3158f020-696e-4d3f-992c-054efbaa67dc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.426197 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.426107 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9668d2df-a646-45c2-b675-6b9a0b4586d5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.426197 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.426156 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9668d2df-a646-45c2-b675-6b9a0b4586d5-node-exporter-accelerators-collector-config\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.426197 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.426191 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9668d2df-a646-45c2-b675-6b9a0b4586d5-node-exporter-wtmp\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.426470 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.426229 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3158f020-696e-4d3f-992c-054efbaa67dc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.426470 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.426432 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9668d2df-a646-45c2-b675-6b9a0b4586d5-node-exporter-wtmp\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.426612 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.426588 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9668d2df-a646-45c2-b675-6b9a0b4586d5-root\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.426701 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.426631 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3158f020-696e-4d3f-992c-054efbaa67dc-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.426701 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.426662 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3158f020-696e-4d3f-992c-054efbaa67dc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.426807 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.426707 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3158f020-696e-4d3f-992c-054efbaa67dc-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.426807 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.426711 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9668d2df-a646-45c2-b675-6b9a0b4586d5-root\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.426807 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.426736 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl47p\" (UniqueName: \"kubernetes.io/projected/3158f020-696e-4d3f-992c-054efbaa67dc-kube-api-access-bl47p\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.426807 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.426781 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9668d2df-a646-45c2-b675-6b9a0b4586d5-node-exporter-tls\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.427037 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.426806 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9668d2df-a646-45c2-b675-6b9a0b4586d5-metrics-client-ca\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.427037 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.426858 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9668d2df-a646-45c2-b675-6b9a0b4586d5-sys\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.427037 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.426884 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9668d2df-a646-45c2-b675-6b9a0b4586d5-node-exporter-textfile\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.427037 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.426911 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3158f020-696e-4d3f-992c-054efbaa67dc-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.427037 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.426936 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v97zv\" (UniqueName: \"kubernetes.io/projected/9668d2df-a646-45c2-b675-6b9a0b4586d5-kube-api-access-v97zv\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.427037 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.427030 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9668d2df-a646-45c2-b675-6b9a0b4586d5-node-exporter-accelerators-collector-config\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.427351 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.427263 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9668d2df-a646-45c2-b675-6b9a0b4586d5-sys\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.427411 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.427394 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3158f020-696e-4d3f-992c-054efbaa67dc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.427976 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.427646 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3158f020-696e-4d3f-992c-054efbaa67dc-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.427976 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:59:33.427662 2580 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 22 02:59:33.427976 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.427644 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9668d2df-a646-45c2-b675-6b9a0b4586d5-node-exporter-textfile\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.427976 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.427718 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9668d2df-a646-45c2-b675-6b9a0b4586d5-metrics-client-ca\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.427976 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:59:33.427726 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3158f020-696e-4d3f-992c-054efbaa67dc-kube-state-metrics-tls podName:3158f020-696e-4d3f-992c-054efbaa67dc nodeName:}" failed. No retries permitted until 2026-07-22 02:59:33.92770737 +0000 UTC m=+103.118901664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3158f020-696e-4d3f-992c-054efbaa67dc-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-4cf24" (UID: "3158f020-696e-4d3f-992c-054efbaa67dc") : secret "kube-state-metrics-tls" not found Jul 22 02:59:33.427976 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.427875 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3158f020-696e-4d3f-992c-054efbaa67dc-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.430650 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.430625 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9668d2df-a646-45c2-b675-6b9a0b4586d5-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.431051 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.431031 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9668d2df-a646-45c2-b675-6b9a0b4586d5-node-exporter-tls\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.431124 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.431074 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3158f020-696e-4d3f-992c-054efbaa67dc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.436211 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.436186 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v97zv\" (UniqueName: \"kubernetes.io/projected/9668d2df-a646-45c2-b675-6b9a0b4586d5-kube-api-access-v97zv\") pod \"node-exporter-f96v5\" (UID: \"9668d2df-a646-45c2-b675-6b9a0b4586d5\") " pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.440942 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.440918 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl47p\" (UniqueName: \"kubernetes.io/projected/3158f020-696e-4d3f-992c-054efbaa67dc-kube-api-access-bl47p\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.532018 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.531974 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-f96v5" Jul 22 02:59:33.931960 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:33.931925 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3158f020-696e-4d3f-992c-054efbaa67dc-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:33.932155 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:59:33.932090 2580 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 22 02:59:33.932226 ip-10-0-138-55 kubenswrapper[2580]: E0722 02:59:33.932162 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3158f020-696e-4d3f-992c-054efbaa67dc-kube-state-metrics-tls podName:3158f020-696e-4d3f-992c-054efbaa67dc nodeName:}" failed. No retries permitted until 2026-07-22 02:59:34.932141464 +0000 UTC m=+104.123335748 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/3158f020-696e-4d3f-992c-054efbaa67dc-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-4cf24" (UID: "3158f020-696e-4d3f-992c-054efbaa67dc") : secret "kube-state-metrics-tls" not found Jul 22 02:59:34.239863 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.239732 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 22 02:59:34.294991 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.294954 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 22 02:59:34.295175 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.295066 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.298100 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.297986 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 22 02:59:34.298245 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.298218 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 22 02:59:34.298245 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.298235 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 22 02:59:34.298871 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.298662 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 22 02:59:34.299111 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.299091 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rccw6\"" Jul 22 02:59:34.299182 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.299162 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 22 02:59:34.299237 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.299167 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 22 02:59:34.299286 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.299093 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 22 02:59:34.299516 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.299500 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 22 02:59:34.299716 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.299565 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 22 02:59:34.435759 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.435725 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-config-volume\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.435759 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.435764 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.436003 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.435801 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.436003 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.435841 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.436003 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.435898 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.436003 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.435940 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.436003 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.435987 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.436247 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.436015 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-config-out\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.436247 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.436037 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-web-config\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.436247 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.436077 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sllbz\" (UniqueName: \"kubernetes.io/projected/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-kube-api-access-sllbz\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.436247 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.436120 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.436247 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.436151 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.436247 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.436191 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.537211 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.537129 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sllbz\" (UniqueName: \"kubernetes.io/projected/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-kube-api-access-sllbz\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.537211 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.537198 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.537429 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.537232 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.537429 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.537281 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.537429 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.537320 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-config-volume\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.537429 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.537346 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.537429 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.537373 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.537429 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.537394 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.537429 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.537430 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.537767 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.537463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.537767 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.537485 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.537767 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.537504 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-config-out\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.537767 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.537526 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-web-config\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.537767 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.537640 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.538742 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.538477 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.539590 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.539564 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.542160 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.541997 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.542268 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.542244 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-config-out\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.542688 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.542657 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.542786 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.542709 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.543016 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.542971 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-web-config\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.543102 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.543078 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.543210 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.543186 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.543574 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.543534 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.544614 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.544563 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-config-volume\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.546961 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.546936 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sllbz\" (UniqueName: \"kubernetes.io/projected/3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b-kube-api-access-sllbz\") pod \"alertmanager-main-0\" (UID: \"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.607592 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.607558 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 22 02:59:34.767263 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.767230 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-szjw4" Jul 22 02:59:34.940645 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.940611 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3158f020-696e-4d3f-992c-054efbaa67dc-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:34.943296 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:34.943271 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3158f020-696e-4d3f-992c-054efbaa67dc-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-4cf24\" (UID: \"3158f020-696e-4d3f-992c-054efbaa67dc\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:35.016370 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:35.016325 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" Jul 22 02:59:36.495861 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:36.495812 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-697458475c-6xmhl"] Jul 22 02:59:36.976883 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:59:36.976842 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9668d2df_a646_45c2_b675_6b9a0b4586d5.slice/crio-ebd1a4d1a8751f689e64d0878814fd80e7097ad0f49d1a502313394657b82c11 WatchSource:0}: Error finding container ebd1a4d1a8751f689e64d0878814fd80e7097ad0f49d1a502313394657b82c11: Status 404 returned error can't find the container with id ebd1a4d1a8751f689e64d0878814fd80e7097ad0f49d1a502313394657b82c11 Jul 22 02:59:37.118031 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:37.118004 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 22 02:59:37.132509 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:37.132488 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-4cf24"] Jul 22 02:59:37.151132 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:37.151100 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-984955c4f-ff8rk"] Jul 22 02:59:37.205351 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:59:37.205316 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e0f600b_5a7f_4c72_addc_e7f83f9b3a1b.slice/crio-9dbdbb32cd57a73c0128ae900f4afdcce0bc8cbb71d6acae0cda3f5e504ba3d5 WatchSource:0}: Error finding container 9dbdbb32cd57a73c0128ae900f4afdcce0bc8cbb71d6acae0cda3f5e504ba3d5: Status 404 returned error can't find the container with id 9dbdbb32cd57a73c0128ae900f4afdcce0bc8cbb71d6acae0cda3f5e504ba3d5 Jul 22 02:59:37.205902 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:59:37.205877 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3158f020_696e_4d3f_992c_054efbaa67dc.slice/crio-5e589f070f5626bfb18087c6553d164b29876afffd55d13e94ed4a44fc8ac549 WatchSource:0}: Error finding container 5e589f070f5626bfb18087c6553d164b29876afffd55d13e94ed4a44fc8ac549: Status 404 returned error can't find the container with id 5e589f070f5626bfb18087c6553d164b29876afffd55d13e94ed4a44fc8ac549 Jul 22 02:59:37.207372 ip-10-0-138-55 kubenswrapper[2580]: W0722 02:59:37.207292 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01f1c1c1_3d02_4a21_8468_4c8c30b91c5a.slice/crio-7d178911147ad429ad664b3e18c7b3bb47c39e21e8aa263df1f191a54594dcd3 WatchSource:0}: Error finding container 7d178911147ad429ad664b3e18c7b3bb47c39e21e8aa263df1f191a54594dcd3: Status 404 returned error can't find the container with id 7d178911147ad429ad664b3e18c7b3bb47c39e21e8aa263df1f191a54594dcd3 Jul 22 02:59:37.934196 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:37.934082 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-984955c4f-ff8rk" event={"ID":"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a","Type":"ContainerStarted","Data":"7c4e7038a246c41e355446033950635d5a4a6100a30969f385269bc47a62979a"} Jul 22 02:59:37.934196 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:37.934140 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-984955c4f-ff8rk" event={"ID":"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a","Type":"ContainerStarted","Data":"7d178911147ad429ad664b3e18c7b3bb47c39e21e8aa263df1f191a54594dcd3"} Jul 22 02:59:37.934196 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:37.934166 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 02:59:37.936737 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:37.936592 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-vd8sx" event={"ID":"8c542afe-969b-42c5-9dc5-dc4eb2586201","Type":"ContainerStarted","Data":"1f6c5adf094c621aa3e1cfdb89d8b7ae1191b915629a509034139330a76c22de"} Jul 22 02:59:37.937247 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:37.937115 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-vd8sx" Jul 22 02:59:37.939821 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:37.939763 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f96v5" event={"ID":"9668d2df-a646-45c2-b675-6b9a0b4586d5","Type":"ContainerStarted","Data":"ebd1a4d1a8751f689e64d0878814fd80e7097ad0f49d1a502313394657b82c11"} Jul 22 02:59:37.941285 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:37.941210 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b","Type":"ContainerStarted","Data":"9dbdbb32cd57a73c0128ae900f4afdcce0bc8cbb71d6acae0cda3f5e504ba3d5"} Jul 22 02:59:37.942716 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:37.942543 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" event={"ID":"3158f020-696e-4d3f-992c-054efbaa67dc","Type":"ContainerStarted","Data":"5e589f070f5626bfb18087c6553d164b29876afffd55d13e94ed4a44fc8ac549"} Jul 22 02:59:37.948702 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:37.948677 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-vd8sx" Jul 22 02:59:37.959471 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:37.959348 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-984955c4f-ff8rk" podStartSLOduration=106.959328422 podStartE2EDuration="1m46.959328422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:57:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:59:37.958282139 +0000 UTC m=+107.149476443" watchObservedRunningTime="2026-07-22 02:59:37.959328422 +0000 UTC m=+107.150522726" Jul 22 02:59:37.978473 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:37.976661 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-vd8sx" podStartSLOduration=1.768125835 podStartE2EDuration="19.976643019s" totalImagesPullingTime="18.208517184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:59:37.97632532 +0000 UTC m=+107.167519623" watchObservedRunningTime="2026-07-22 02:59:37.976643019 +0000 UTC m=+107.167837323" Jul 22 02:59:39.390843 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.390790 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 02:59:39.415105 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.413177 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.421270 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.420552 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 22 02:59:39.421270 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.420684 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 22 02:59:39.421270 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.420967 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 22 02:59:39.421270 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.421210 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-ak53kfkdu28l6\"" Jul 22 02:59:39.421587 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.421400 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 22 02:59:39.421587 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.421582 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jvf5n\"" Jul 22 02:59:39.422982 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.421758 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 22 02:59:39.422982 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.422308 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 22 02:59:39.422982 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.422518 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 22 02:59:39.422982 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.422732 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 22 02:59:39.422982 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.422932 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 22 02:59:39.423275 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.423155 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 22 02:59:39.426347 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.424625 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 22 02:59:39.426347 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.424779 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 22 02:59:39.426347 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.425056 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 22 02:59:39.426722 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.426605 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 02:59:39.427071 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.427054 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 22 02:59:39.430469 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.430403 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 22 02:59:39.485204 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485157 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.485377 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485214 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcrc5\" (UniqueName: \"kubernetes.io/projected/813c34cd-40f0-4be0-a8f1-b943263a093b-kube-api-access-vcrc5\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.485377 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485253 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.485377 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485288 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.485377 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485308 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.485377 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485361 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.485624 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485405 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.485624 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485430 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.485624 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485471 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-config\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.485624 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485496 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.485624 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485514 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.485624 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485531 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/813c34cd-40f0-4be0-a8f1-b943263a093b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.485624 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485545 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/813c34cd-40f0-4be0-a8f1-b943263a093b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.485624 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485589 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/813c34cd-40f0-4be0-a8f1-b943263a093b-config-out\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.485624 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485618 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-web-config\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.486013 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485693 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.486013 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485721 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.486013 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485745 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.486013 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485770 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.486013 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.485794 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.586905 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.586867 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/813c34cd-40f0-4be0-a8f1-b943263a093b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.586905 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.586907 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/813c34cd-40f0-4be0-a8f1-b943263a093b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587152 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587042 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/813c34cd-40f0-4be0-a8f1-b943263a093b-config-out\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587152 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587084 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-web-config\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587271 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587160 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587271 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587191 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587271 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587218 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587409 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587302 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587409 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587409 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587378 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587409 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587405 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcrc5\" (UniqueName: \"kubernetes.io/projected/813c34cd-40f0-4be0-a8f1-b943263a093b-kube-api-access-vcrc5\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587610 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587442 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587610 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587491 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587610 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587520 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587610 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587551 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587610 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587575 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587610 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587602 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587985 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587655 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-config\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587985 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587680 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587985 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587697 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/813c34cd-40f0-4be0-a8f1-b943263a093b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.587985 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.587703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.588438 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.588330 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.588625 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.588599 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.589699 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.588713 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.589699 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.589343 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.590229 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.589898 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.591092 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.591072 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.592517 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.592485 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.593574 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.593288 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-config\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.593574 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.593522 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/813c34cd-40f0-4be0-a8f1-b943263a093b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.593781 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.593756 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.594326 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.594271 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/813c34cd-40f0-4be0-a8f1-b943263a093b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.594917 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.594868 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/813c34cd-40f0-4be0-a8f1-b943263a093b-config-out\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.595801 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.595244 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.595801 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.595426 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-web-config\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.595801 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.595773 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.595801 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.595771 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.596131 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.596101 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.596767 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.596742 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/813c34cd-40f0-4be0-a8f1-b943263a093b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.598521 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.598500 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcrc5\" (UniqueName: \"kubernetes.io/projected/813c34cd-40f0-4be0-a8f1-b943263a093b-kube-api-access-vcrc5\") pod \"prometheus-k8s-0\" (UID: \"813c34cd-40f0-4be0-a8f1-b943263a093b\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:39.731981 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:39.731904 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:40.198363 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:40.198209 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 22 02:59:40.958004 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:40.957967 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" event={"ID":"3158f020-696e-4d3f-992c-054efbaa67dc","Type":"ContainerStarted","Data":"5babfb3d3f5865e5da12c78c1ea0304627ab7d09fdc81da1c98f0dbd38c849fe"} Jul 22 02:59:40.958558 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:40.958044 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" event={"ID":"3158f020-696e-4d3f-992c-054efbaa67dc","Type":"ContainerStarted","Data":"5a6e2a6a5306643aa55ee327611b2588bcd18e3ba37e17bb2d1945616cd1944c"} Jul 22 02:59:40.958558 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:40.958082 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" event={"ID":"3158f020-696e-4d3f-992c-054efbaa67dc","Type":"ContainerStarted","Data":"2a4886806883efb5d51a2014da6e9d2cec1c7f17e1db8d3711ee075ee92b4e61"} Jul 22 02:59:40.959976 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:40.959946 2580 generic.go:356] "Generic (PLEG): container finished" podID="9668d2df-a646-45c2-b675-6b9a0b4586d5" containerID="d17591c5d77b1c9c21a9f97b53f53f8f3c27c94dae8137b9b173b7b0aa9222f9" exitCode=0 Jul 22 02:59:40.960098 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:40.960036 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f96v5" event={"ID":"9668d2df-a646-45c2-b675-6b9a0b4586d5","Type":"ContainerDied","Data":"d17591c5d77b1c9c21a9f97b53f53f8f3c27c94dae8137b9b173b7b0aa9222f9"} Jul 22 02:59:40.962148 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:40.962121 2580 generic.go:356] "Generic (PLEG): container finished" podID="3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b" containerID="aef3a4dd38b726bfe5ade82074b46664105ffbd21d17947e3707ab56f69b8754" exitCode=0 Jul 22 02:59:40.962267 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:40.962151 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b","Type":"ContainerDied","Data":"aef3a4dd38b726bfe5ade82074b46664105ffbd21d17947e3707ab56f69b8754"} Jul 22 02:59:40.964120 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:40.964094 2580 generic.go:356] "Generic (PLEG): container finished" podID="813c34cd-40f0-4be0-a8f1-b943263a093b" containerID="3145a724ccca905e1ae054d81bc0039d67937dad915ec26d7cab7e9c901212fc" exitCode=0 Jul 22 02:59:40.964214 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:40.964147 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"813c34cd-40f0-4be0-a8f1-b943263a093b","Type":"ContainerDied","Data":"3145a724ccca905e1ae054d81bc0039d67937dad915ec26d7cab7e9c901212fc"} Jul 22 02:59:40.964214 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:40.964209 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"813c34cd-40f0-4be0-a8f1-b943263a093b","Type":"ContainerStarted","Data":"b945ec693c400f22783b6d06dcf952b24eec234b2b9d156f7ba03020d247cb18"} Jul 22 02:59:40.981482 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:40.981437 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-4cf24" podStartSLOduration=5.179423165 podStartE2EDuration="7.981419713s" totalImagesPullingTime="2.801996548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:59:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:59:40.980048022 +0000 UTC m=+110.171242344" watchObservedRunningTime="2026-07-22 02:59:40.981419713 +0000 UTC m=+110.172614015" Jul 22 02:59:41.977981 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:41.977879 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f96v5" event={"ID":"9668d2df-a646-45c2-b675-6b9a0b4586d5","Type":"ContainerStarted","Data":"9ffc04952461e8ccaef545a66ae30d92a4cff1e50d7f81e82ce840cc07b42ce1"} Jul 22 02:59:41.977981 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:41.977945 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-f96v5" event={"ID":"9668d2df-a646-45c2-b675-6b9a0b4586d5","Type":"ContainerStarted","Data":"5384c44ca77d96e47544d78118bbf51720caff26c7221b99d3235df5c3b546b4"} Jul 22 02:59:42.397670 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:42.397604 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-f96v5" podStartSLOduration=6.26497632 podStartE2EDuration="9.397586296s" totalImagesPullingTime="3.006422543s" totalInitContainerRuntime="126.187433ms" isStatefulPod=true podCreationTimestamp="2026-07-22 02:59:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:59:42.002353861 +0000 UTC m=+111.193548165" watchObservedRunningTime="2026-07-22 02:59:42.397586296 +0000 UTC m=+111.588780599" Jul 22 02:59:42.398525 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:42.398482 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-984955c4f-ff8rk"] Jul 22 02:59:43.992378 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:43.992088 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b","Type":"ContainerStarted","Data":"924f4bca3c5ebe47e5ca29d61e223b9dd6e599193ebcfccc690d5bb6543819a1"} Jul 22 02:59:43.992378 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:43.992214 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b","Type":"ContainerStarted","Data":"9a3f2f4dc1605bb188963a91be72d337c0047a7da92f2797323a6afa1bec6e78"} Jul 22 02:59:46.004509 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:46.004464 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"813c34cd-40f0-4be0-a8f1-b943263a093b","Type":"ContainerStarted","Data":"04de27f4a47332bef0ad8f063c4626010281224a70c83cf6f35fadfd11be850e"} Jul 22 02:59:46.005047 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:46.004556 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"813c34cd-40f0-4be0-a8f1-b943263a093b","Type":"ContainerStarted","Data":"56f70d20b4e0c57165b06f92f62cb4edf98e48936c8045bfa5d7432f9328edd5"} Jul 22 02:59:46.009254 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:46.009220 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b","Type":"ContainerStarted","Data":"24d2cb16fb59d6581d5f959bc06332d1922fd7c38ce18a41bfdac0b1533416d7"} Jul 22 02:59:46.009402 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:46.009294 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b","Type":"ContainerStarted","Data":"c633c79d3c7e9260f317228e4d6064771f4fe6402ff5c99351d66fbb47bc7157"} Jul 22 02:59:46.009402 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:46.009334 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b","Type":"ContainerStarted","Data":"caa5e567d001108dae03e6591edfccadc6aa8ecfcb8c3c95f9741a4f1c6ef3fb"} Jul 22 02:59:47.020210 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:47.020126 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b","Type":"ContainerStarted","Data":"10e1ca25cdd16a537cb7fed38181a2de7e08873bbedfc5cb43386e5443448382"} Jul 22 02:59:47.051994 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:47.051932 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=6.559843019 podStartE2EDuration="13.05191031s" totalImagesPullingTime="6.464173931s" totalInitContainerRuntime="27.89336ms" isStatefulPod=false podCreationTimestamp="2026-07-22 02:59:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:59:47.049274312 +0000 UTC m=+116.240468608" watchObservedRunningTime="2026-07-22 02:59:47.05191031 +0000 UTC m=+116.243104613" Jul 22 02:59:48.029631 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:48.029597 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"813c34cd-40f0-4be0-a8f1-b943263a093b","Type":"ContainerStarted","Data":"a9b25a1e74e154983d9bc16a6c20fedbd8b3f992aa53e3db4a884ca747eb9767"} Jul 22 02:59:49.036361 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:49.036328 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"813c34cd-40f0-4be0-a8f1-b943263a093b","Type":"ContainerStarted","Data":"3cd7ad8ee45f63a9ddec1dce659969465d3961e5709c4cd9c9ac05371002c1b2"} Jul 22 02:59:49.036741 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:49.036393 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"813c34cd-40f0-4be0-a8f1-b943263a093b","Type":"ContainerStarted","Data":"feac32e36525d4f93974aac94ac464988488bc98726f9d515ec7d93acbd16274"} Jul 22 02:59:49.036741 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:49.036425 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"813c34cd-40f0-4be0-a8f1-b943263a093b","Type":"ContainerStarted","Data":"5e2f1645789291ddf26813d470910abae000260188bdba03752efeafa08505b7"} Jul 22 02:59:49.068199 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:49.068148 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.4713164389999998 podStartE2EDuration="10.068132184s" totalImagesPullingTime="6.566481805s" totalInitContainerRuntime="30.33394ms" isStatefulPod=false podCreationTimestamp="2026-07-22 02:59:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:59:49.066679038 +0000 UTC m=+118.257873342" watchObservedRunningTime="2026-07-22 02:59:49.068132184 +0000 UTC m=+118.259326488" Jul 22 02:59:49.732988 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:49.732958 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 02:59:52.404524 ip-10-0-138-55 kubenswrapper[2580]: I0722 02:59:52.404495 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 03:00:01.093814 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.093778 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs\") pod \"network-metrics-daemon-mlxzc\" (UID: \"31d06428-0968-400f-8b93-86bacf6ec123\") " pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 03:00:01.096122 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.096103 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/31d06428-0968-400f-8b93-86bacf6ec123-metrics-certs\") pod \"network-metrics-daemon-mlxzc\" (UID: \"31d06428-0968-400f-8b93-86bacf6ec123\") " pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 03:00:01.275729 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.275697 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sqq4d\"" Jul 22 03:00:01.283775 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.283753 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlxzc" Jul 22 03:00:01.444738 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.444615 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mlxzc"] Jul 22 03:00:01.447597 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:00:01.447567 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31d06428_0968_400f_8b93_86bacf6ec123.slice/crio-a0feedaf451a97812a0ab3877d95c1f272e25b222aa503a52cd60d33be8125d0 WatchSource:0}: Error finding container a0feedaf451a97812a0ab3877d95c1f272e25b222aa503a52cd60d33be8125d0: Status 404 returned error can't find the container with id a0feedaf451a97812a0ab3877d95c1f272e25b222aa503a52cd60d33be8125d0 Jul 22 03:00:01.520246 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.520195 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-697458475c-6xmhl" podUID="e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0" containerName="console" containerID="cri-o://eb9487d356d90b930c04f449735a915a7465876920e315fcac2b675c384a76d5" gracePeriod=15 Jul 22 03:00:01.580689 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:00:01.580657 2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode75b4bc8_9ca1_4bbe_b91d_35b32dda2ad0.slice/crio-conmon-eb9487d356d90b930c04f449735a915a7465876920e315fcac2b675c384a76d5.scope\": RecentStats: unable to find data in memory cache]" Jul 22 03:00:01.792990 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.792959 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697458475c-6xmhl" Jul 22 03:00:01.899823 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.899789 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-config\") pod \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " Jul 22 03:00:01.900025 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.899856 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-service-ca\") pod \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " Jul 22 03:00:01.900025 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.899883 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-oauth-config\") pod \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " Jul 22 03:00:01.900025 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.899941 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjd5c\" (UniqueName: \"kubernetes.io/projected/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-kube-api-access-kjd5c\") pod \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " Jul 22 03:00:01.900186 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.900036 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-serving-cert\") pod \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " Jul 22 03:00:01.900186 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.900061 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-oauth-serving-cert\") pod \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\" (UID: \"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0\") " Jul 22 03:00:01.900284 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.900235 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-config" (OuterVolumeSpecName: "console-config") pod "e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0" (UID: "e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:00:01.900374 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.900355 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-config\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:00:01.900430 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.900351 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-service-ca" (OuterVolumeSpecName: "service-ca") pod "e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0" (UID: "e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:00:01.900515 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.900496 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0" (UID: "e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:00:01.902218 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.902192 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0" (UID: "e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:00:01.902381 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.902358 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-kube-api-access-kjd5c" (OuterVolumeSpecName: "kube-api-access-kjd5c") pod "e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0" (UID: "e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0"). InnerVolumeSpecName "kube-api-access-kjd5c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:00:01.902435 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:01.902375 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0" (UID: "e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:00:02.001045 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:02.000952 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjd5c\" (UniqueName: \"kubernetes.io/projected/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-kube-api-access-kjd5c\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:00:02.001045 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:02.000989 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-serving-cert\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:00:02.001045 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:02.000999 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-oauth-serving-cert\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:00:02.001045 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:02.001008 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-service-ca\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:00:02.001045 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:02.001019 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0-console-oauth-config\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:00:02.090621 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:02.090588 2580 generic.go:356] "Generic (PLEG): container finished" podID="e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0" containerID="eb9487d356d90b930c04f449735a915a7465876920e315fcac2b675c384a76d5" exitCode=0 Jul 22 03:00:02.090875 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:02.090630 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-697458475c-6xmhl" Jul 22 03:00:02.090875 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:02.090682 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697458475c-6xmhl" event={"ID":"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0","Type":"ContainerDied","Data":"eb9487d356d90b930c04f449735a915a7465876920e315fcac2b675c384a76d5"} Jul 22 03:00:02.090875 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:02.090737 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-697458475c-6xmhl" event={"ID":"e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0","Type":"ContainerDied","Data":"5b4057fbaec839fe134fe624d2363fd835770f0777cba09e9cf61838d61fae5a"} Jul 22 03:00:02.090875 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:02.090765 2580 scope.go:117] "RemoveContainer" containerID="eb9487d356d90b930c04f449735a915a7465876920e315fcac2b675c384a76d5" Jul 22 03:00:02.091972 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:02.091953 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mlxzc" event={"ID":"31d06428-0968-400f-8b93-86bacf6ec123","Type":"ContainerStarted","Data":"a0feedaf451a97812a0ab3877d95c1f272e25b222aa503a52cd60d33be8125d0"} Jul 22 03:00:02.099508 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:02.099493 2580 scope.go:117] "RemoveContainer" containerID="eb9487d356d90b930c04f449735a915a7465876920e315fcac2b675c384a76d5" Jul 22 03:00:02.099791 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:00:02.099747 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb9487d356d90b930c04f449735a915a7465876920e315fcac2b675c384a76d5\": container with ID starting with eb9487d356d90b930c04f449735a915a7465876920e315fcac2b675c384a76d5 not found: ID does not exist" containerID="eb9487d356d90b930c04f449735a915a7465876920e315fcac2b675c384a76d5" Jul 22 03:00:02.099863 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:02.099783 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb9487d356d90b930c04f449735a915a7465876920e315fcac2b675c384a76d5"} err="failed to get container status \"eb9487d356d90b930c04f449735a915a7465876920e315fcac2b675c384a76d5\": rpc error: code = NotFound desc = could not find container \"eb9487d356d90b930c04f449735a915a7465876920e315fcac2b675c384a76d5\": container with ID starting with eb9487d356d90b930c04f449735a915a7465876920e315fcac2b675c384a76d5 not found: ID does not exist" Jul 22 03:00:02.122868 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:02.122848 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-697458475c-6xmhl"] Jul 22 03:00:02.125297 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:02.125278 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-697458475c-6xmhl"] Jul 22 03:00:03.367468 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:03.367432 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0" path="/var/lib/kubelet/pods/e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0/volumes" Jul 22 03:00:05.107175 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:05.107136 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mlxzc" event={"ID":"31d06428-0968-400f-8b93-86bacf6ec123","Type":"ContainerStarted","Data":"c6bbabf291db5e44c120b07609b18278182a3ac14a82630620a9c94ac1eeedc1"} Jul 22 03:00:05.107529 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:05.107224 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mlxzc" event={"ID":"31d06428-0968-400f-8b93-86bacf6ec123","Type":"ContainerStarted","Data":"32268be9c4873faa3ac9a87798254816304b30e3862cec04addae7652613507d"} Jul 22 03:00:05.133663 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:05.133606 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mlxzc" podStartSLOduration=131.492684833 podStartE2EDuration="2m14.13359091s" totalImagesPullingTime="2.640906077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:57:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:00:05.132497465 +0000 UTC m=+134.323691786" watchObservedRunningTime="2026-07-22 03:00:05.13359091 +0000 UTC m=+134.324785213" Jul 22 03:00:07.428106 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.428060 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-984955c4f-ff8rk" podUID="01f1c1c1-3d02-4a21-8468-4c8c30b91c5a" containerName="registry" containerID="cri-o://7c4e7038a246c41e355446033950635d5a4a6100a30969f385269bc47a62979a" gracePeriod=30 Jul 22 03:00:07.708155 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.708078 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 03:00:07.859175 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.859137 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prlgm\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-kube-api-access-prlgm\") pod \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " Jul 22 03:00:07.859365 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.859197 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-installation-pull-secrets\") pod \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " Jul 22 03:00:07.859365 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.859227 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-ca-trust-extracted\") pod \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " Jul 22 03:00:07.859365 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.859245 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-certificates\") pod \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " Jul 22 03:00:07.859365 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.859266 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-image-registry-private-configuration\") pod \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " Jul 22 03:00:07.859572 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.859431 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls\") pod \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " Jul 22 03:00:07.859572 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.859495 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-bound-sa-token\") pod \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " Jul 22 03:00:07.859572 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.859532 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-trusted-ca\") pod \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\" (UID: \"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a\") " Jul 22 03:00:07.860101 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.859928 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a" (UID: "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:00:07.860221 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.860159 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a" (UID: "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:00:07.862035 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.861976 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a" (UID: "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:00:07.862168 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.862143 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-kube-api-access-prlgm" (OuterVolumeSpecName: "kube-api-access-prlgm") pod "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a" (UID: "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a"). InnerVolumeSpecName "kube-api-access-prlgm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:00:07.862168 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.862156 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a" (UID: "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:00:07.862330 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.862146 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a" (UID: "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:00:07.862330 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.862237 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a" (UID: "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:00:07.868129 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.868105 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a" (UID: "01f1c1c1-3d02-4a21-8468-4c8c30b91c5a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:00:07.960503 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.960415 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-bound-sa-token\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:00:07.960503 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.960447 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-trusted-ca\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:00:07.960503 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.960456 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prlgm\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-kube-api-access-prlgm\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:00:07.960503 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.960466 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-installation-pull-secrets\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:00:07.960503 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.960476 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-ca-trust-extracted\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:00:07.960503 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.960485 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-certificates\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:00:07.960503 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.960496 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-image-registry-private-configuration\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:00:07.960503 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:07.960504 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a-registry-tls\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:00:08.120379 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:08.120341 2580 generic.go:356] "Generic (PLEG): container finished" podID="01f1c1c1-3d02-4a21-8468-4c8c30b91c5a" containerID="7c4e7038a246c41e355446033950635d5a4a6100a30969f385269bc47a62979a" exitCode=0 Jul 22 03:00:08.120564 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:08.120385 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-984955c4f-ff8rk" Jul 22 03:00:08.120564 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:08.120388 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-984955c4f-ff8rk" event={"ID":"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a","Type":"ContainerDied","Data":"7c4e7038a246c41e355446033950635d5a4a6100a30969f385269bc47a62979a"} Jul 22 03:00:08.120564 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:08.120499 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-984955c4f-ff8rk" event={"ID":"01f1c1c1-3d02-4a21-8468-4c8c30b91c5a","Type":"ContainerDied","Data":"7d178911147ad429ad664b3e18c7b3bb47c39e21e8aa263df1f191a54594dcd3"} Jul 22 03:00:08.120564 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:08.120526 2580 scope.go:117] "RemoveContainer" containerID="7c4e7038a246c41e355446033950635d5a4a6100a30969f385269bc47a62979a" Jul 22 03:00:08.133770 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:08.133748 2580 scope.go:117] "RemoveContainer" containerID="7c4e7038a246c41e355446033950635d5a4a6100a30969f385269bc47a62979a" Jul 22 03:00:08.134086 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:00:08.134069 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c4e7038a246c41e355446033950635d5a4a6100a30969f385269bc47a62979a\": container with ID starting with 7c4e7038a246c41e355446033950635d5a4a6100a30969f385269bc47a62979a not found: ID does not exist" containerID="7c4e7038a246c41e355446033950635d5a4a6100a30969f385269bc47a62979a" Jul 22 03:00:08.134185 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:08.134101 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c4e7038a246c41e355446033950635d5a4a6100a30969f385269bc47a62979a"} err="failed to get container status \"7c4e7038a246c41e355446033950635d5a4a6100a30969f385269bc47a62979a\": rpc error: code = NotFound desc = could not find container \"7c4e7038a246c41e355446033950635d5a4a6100a30969f385269bc47a62979a\": container with ID starting with 7c4e7038a246c41e355446033950635d5a4a6100a30969f385269bc47a62979a not found: ID does not exist" Jul 22 03:00:08.145006 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:08.144976 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-984955c4f-ff8rk"] Jul 22 03:00:08.148525 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:08.148496 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-984955c4f-ff8rk"] Jul 22 03:00:09.365821 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:09.365784 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01f1c1c1-3d02-4a21-8468-4c8c30b91c5a" path="/var/lib/kubelet/pods/01f1c1c1-3d02-4a21-8468-4c8c30b91c5a/volumes" Jul 22 03:00:39.732589 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:39.732553 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 03:00:39.751931 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:39.751903 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 03:00:40.261629 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:00:40.261604 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 22 03:02:38.999964 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:38.999931 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jkrs7"] Jul 22 03:02:39.000383 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.000240 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0" containerName="console" Jul 22 03:02:39.000383 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.000252 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0" containerName="console" Jul 22 03:02:39.000383 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.000261 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01f1c1c1-3d02-4a21-8468-4c8c30b91c5a" containerName="registry" Jul 22 03:02:39.000383 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.000267 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="01f1c1c1-3d02-4a21-8468-4c8c30b91c5a" containerName="registry" Jul 22 03:02:39.000383 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.000331 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e75b4bc8-9ca1-4bbe-b91d-35b32dda2ad0" containerName="console" Jul 22 03:02:39.000383 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.000342 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="01f1c1c1-3d02-4a21-8468-4c8c30b91c5a" containerName="registry" Jul 22 03:02:39.003153 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.003138 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jkrs7" Jul 22 03:02:39.005821 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.005798 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 22 03:02:39.006607 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.006589 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 22 03:02:39.006672 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.006589 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-7pq9s\"" Jul 22 03:02:39.026599 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.026565 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jkrs7"] Jul 22 03:02:39.100116 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.100083 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chvzt\" (UniqueName: \"kubernetes.io/projected/b2c39a89-f0f9-43c4-af36-26578cc8e90d-kube-api-access-chvzt\") pod \"openshift-lws-operator-fd8ccff4c-jkrs7\" (UID: \"b2c39a89-f0f9-43c4-af36-26578cc8e90d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jkrs7" Jul 22 03:02:39.100312 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.100124 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2c39a89-f0f9-43c4-af36-26578cc8e90d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jkrs7\" (UID: \"b2c39a89-f0f9-43c4-af36-26578cc8e90d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jkrs7" Jul 22 03:02:39.201267 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.201230 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chvzt\" (UniqueName: \"kubernetes.io/projected/b2c39a89-f0f9-43c4-af36-26578cc8e90d-kube-api-access-chvzt\") pod \"openshift-lws-operator-fd8ccff4c-jkrs7\" (UID: \"b2c39a89-f0f9-43c4-af36-26578cc8e90d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jkrs7" Jul 22 03:02:39.201449 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.201277 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2c39a89-f0f9-43c4-af36-26578cc8e90d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jkrs7\" (UID: \"b2c39a89-f0f9-43c4-af36-26578cc8e90d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jkrs7" Jul 22 03:02:39.201700 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.201680 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b2c39a89-f0f9-43c4-af36-26578cc8e90d-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jkrs7\" (UID: \"b2c39a89-f0f9-43c4-af36-26578cc8e90d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jkrs7" Jul 22 03:02:39.212360 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.212335 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chvzt\" (UniqueName: \"kubernetes.io/projected/b2c39a89-f0f9-43c4-af36-26578cc8e90d-kube-api-access-chvzt\") pod \"openshift-lws-operator-fd8ccff4c-jkrs7\" (UID: \"b2c39a89-f0f9-43c4-af36-26578cc8e90d\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jkrs7" Jul 22 03:02:39.320738 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.320639 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jkrs7" Jul 22 03:02:39.456028 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.455998 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jkrs7"] Jul 22 03:02:39.459422 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:02:39.459395 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2c39a89_f0f9_43c4_af36_26578cc8e90d.slice/crio-91c14c41e1bb878a27c7efa2e49c7403c63e66fdf2b3910b600ade9d03823203 WatchSource:0}: Error finding container 91c14c41e1bb878a27c7efa2e49c7403c63e66fdf2b3910b600ade9d03823203: Status 404 returned error can't find the container with id 91c14c41e1bb878a27c7efa2e49c7403c63e66fdf2b3910b600ade9d03823203 Jul 22 03:02:39.699731 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:39.699697 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jkrs7" event={"ID":"b2c39a89-f0f9-43c4-af36-26578cc8e90d","Type":"ContainerStarted","Data":"91c14c41e1bb878a27c7efa2e49c7403c63e66fdf2b3910b600ade9d03823203"} Jul 22 03:02:42.714668 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:42.714625 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jkrs7" event={"ID":"b2c39a89-f0f9-43c4-af36-26578cc8e90d","Type":"ContainerStarted","Data":"2b62b8324c9364a3f8c437be95aa67529c78e516418ef3b987c1fcddbbc2d0a7"} Jul 22 03:02:42.736708 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:42.736648 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jkrs7" podStartSLOduration=2.055786463 podStartE2EDuration="4.736629228s" totalImagesPullingTime="2.680842765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:02:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:02:42.734848181 +0000 UTC m=+291.926042482" watchObservedRunningTime="2026-07-22 03:02:42.736629228 +0000 UTC m=+291.927823531" Jul 22 03:02:51.214930 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:51.214896 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:02:51.223615 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:51.223589 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:02:51.224963 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:51.224940 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:02:51.228917 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:51.228889 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 03:02:51.231019 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:02:51.231001 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:03:02.696525 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.696489 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z"] Jul 22 03:03:02.699996 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.699969 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:02.705254 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.705229 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 03:03:02.705507 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.705490 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 03:03:02.705593 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.705528 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-f4xjc\"" Jul 22 03:03:02.705593 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.705566 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 03:03:02.705804 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.705790 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 03:03:02.728002 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.727971 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z"] Jul 22 03:03:02.799601 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.799573 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f0fe9a9a-4f8a-4b1a-bf18-70417a38239c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94fc48448-chs9z\" (UID: \"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c\") " pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:02.799809 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.799610 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0fe9a9a-4f8a-4b1a-bf18-70417a38239c-tmp\") pod \"opendatahub-operator-controller-manager-94fc48448-chs9z\" (UID: \"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c\") " pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:02.799809 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.799686 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87p7q\" (UniqueName: \"kubernetes.io/projected/f0fe9a9a-4f8a-4b1a-bf18-70417a38239c-kube-api-access-87p7q\") pod \"opendatahub-operator-controller-manager-94fc48448-chs9z\" (UID: \"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c\") " pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:02.799809 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.799734 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f0fe9a9a-4f8a-4b1a-bf18-70417a38239c-manifests\") pod \"opendatahub-operator-controller-manager-94fc48448-chs9z\" (UID: \"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c\") " pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:02.799809 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.799777 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0fe9a9a-4f8a-4b1a-bf18-70417a38239c-webhook-cert\") pod \"opendatahub-operator-controller-manager-94fc48448-chs9z\" (UID: \"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c\") " pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:02.900246 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.900208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87p7q\" (UniqueName: \"kubernetes.io/projected/f0fe9a9a-4f8a-4b1a-bf18-70417a38239c-kube-api-access-87p7q\") pod \"opendatahub-operator-controller-manager-94fc48448-chs9z\" (UID: \"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c\") " pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:02.900246 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.900249 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f0fe9a9a-4f8a-4b1a-bf18-70417a38239c-manifests\") pod \"opendatahub-operator-controller-manager-94fc48448-chs9z\" (UID: \"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c\") " pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:02.900453 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.900273 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0fe9a9a-4f8a-4b1a-bf18-70417a38239c-webhook-cert\") pod \"opendatahub-operator-controller-manager-94fc48448-chs9z\" (UID: \"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c\") " pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:02.900453 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.900311 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f0fe9a9a-4f8a-4b1a-bf18-70417a38239c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94fc48448-chs9z\" (UID: \"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c\") " pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:02.900453 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.900333 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0fe9a9a-4f8a-4b1a-bf18-70417a38239c-tmp\") pod \"opendatahub-operator-controller-manager-94fc48448-chs9z\" (UID: \"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c\") " pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:02.900754 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.900733 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f0fe9a9a-4f8a-4b1a-bf18-70417a38239c-manifests\") pod \"opendatahub-operator-controller-manager-94fc48448-chs9z\" (UID: \"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c\") " pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:02.902747 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.902719 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f0fe9a9a-4f8a-4b1a-bf18-70417a38239c-tmp\") pod \"opendatahub-operator-controller-manager-94fc48448-chs9z\" (UID: \"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c\") " pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:02.902940 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.902922 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f0fe9a9a-4f8a-4b1a-bf18-70417a38239c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-94fc48448-chs9z\" (UID: \"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c\") " pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:02.903002 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.902975 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f0fe9a9a-4f8a-4b1a-bf18-70417a38239c-webhook-cert\") pod \"opendatahub-operator-controller-manager-94fc48448-chs9z\" (UID: \"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c\") " pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:02.916844 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:02.916813 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87p7q\" (UniqueName: \"kubernetes.io/projected/f0fe9a9a-4f8a-4b1a-bf18-70417a38239c-kube-api-access-87p7q\") pod \"opendatahub-operator-controller-manager-94fc48448-chs9z\" (UID: \"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c\") " pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:03.012911 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:03.012797 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:03.151952 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:03.151893 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z"] Jul 22 03:03:03.154750 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:03:03.154726 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0fe9a9a_4f8a_4b1a_bf18_70417a38239c.slice/crio-f43ae6d433150ada1ea5ac15c6b29547238e464dc127725e58ba7f548903da8e WatchSource:0}: Error finding container f43ae6d433150ada1ea5ac15c6b29547238e464dc127725e58ba7f548903da8e: Status 404 returned error can't find the container with id f43ae6d433150ada1ea5ac15c6b29547238e464dc127725e58ba7f548903da8e Jul 22 03:03:03.156811 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:03.156789 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 03:03:03.799649 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:03.799589 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" event={"ID":"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c","Type":"ContainerStarted","Data":"f43ae6d433150ada1ea5ac15c6b29547238e464dc127725e58ba7f548903da8e"} Jul 22 03:03:06.815756 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:06.815721 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" event={"ID":"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c","Type":"ContainerStarted","Data":"b3e0784196096b2f625e2c7a8a92e61e15fc75b6f89767f3bee6844ae741b37d"} Jul 22 03:03:07.821891 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:07.821778 2580 generic.go:356] "Generic (PLEG): container finished" podID="f0fe9a9a-4f8a-4b1a-bf18-70417a38239c" containerID="b3e0784196096b2f625e2c7a8a92e61e15fc75b6f89767f3bee6844ae741b37d" exitCode=0 Jul 22 03:03:07.821891 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:07.821871 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" event={"ID":"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c","Type":"ContainerDied","Data":"b3e0784196096b2f625e2c7a8a92e61e15fc75b6f89767f3bee6844ae741b37d"} Jul 22 03:03:08.827665 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:08.827629 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" event={"ID":"f0fe9a9a-4f8a-4b1a-bf18-70417a38239c","Type":"ContainerStarted","Data":"841309804d7e222495ab0a930a69fa634839fe591affd47fb5200a5d505ce646"} Jul 22 03:03:08.828085 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:08.827744 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:08.866378 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:08.866327 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" podStartSLOduration=2.340422465 podStartE2EDuration="6.866310551s" totalImagesPullingTime="3.462662098s" totalInitContainerRuntime="1.063225988s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:03:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:03:08.865339932 +0000 UTC m=+318.056534234" watchObservedRunningTime="2026-07-22 03:03:08.866310551 +0000 UTC m=+318.057504910" Jul 22 03:03:19.834779 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:19.834743 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-94fc48448-chs9z" Jul 22 03:03:30.492387 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.492352 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8"] Jul 22 03:03:30.503122 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.503091 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.507773 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.507746 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-s9t4w\"" Jul 22 03:03:30.507951 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.507798 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 22 03:03:30.507951 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.507799 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 03:03:30.510259 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.510199 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 22 03:03:30.510783 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.510751 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 22 03:03:30.510783 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.510768 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 03:03:30.510997 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.510771 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 03:03:30.526536 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.526498 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8"] Jul 22 03:03:30.635044 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.635009 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02f86537-3d6d-4b62-a70f-f44d3e6f3537-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.635248 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.635060 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02f86537-3d6d-4b62-a70f-f44d3e6f3537-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.635248 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.635119 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02f86537-3d6d-4b62-a70f-f44d3e6f3537-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.635248 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.635166 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02f86537-3d6d-4b62-a70f-f44d3e6f3537-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.635248 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.635200 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02f86537-3d6d-4b62-a70f-f44d3e6f3537-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.635436 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.635266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02f86537-3d6d-4b62-a70f-f44d3e6f3537-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.635436 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.635298 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctvdc\" (UniqueName: \"kubernetes.io/projected/02f86537-3d6d-4b62-a70f-f44d3e6f3537-kube-api-access-ctvdc\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.736462 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.736424 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02f86537-3d6d-4b62-a70f-f44d3e6f3537-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.736664 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.736473 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02f86537-3d6d-4b62-a70f-f44d3e6f3537-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.736664 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.736507 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02f86537-3d6d-4b62-a70f-f44d3e6f3537-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.736664 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.736524 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02f86537-3d6d-4b62-a70f-f44d3e6f3537-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.736664 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.736545 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02f86537-3d6d-4b62-a70f-f44d3e6f3537-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.736664 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.736579 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02f86537-3d6d-4b62-a70f-f44d3e6f3537-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.736664 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.736607 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctvdc\" (UniqueName: \"kubernetes.io/projected/02f86537-3d6d-4b62-a70f-f44d3e6f3537-kube-api-access-ctvdc\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.737353 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.737329 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/02f86537-3d6d-4b62-a70f-f44d3e6f3537-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.738944 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.738916 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/02f86537-3d6d-4b62-a70f-f44d3e6f3537-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.739033 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.739010 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/02f86537-3d6d-4b62-a70f-f44d3e6f3537-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.739222 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.739203 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/02f86537-3d6d-4b62-a70f-f44d3e6f3537-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.739287 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.739240 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/02f86537-3d6d-4b62-a70f-f44d3e6f3537-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.745324 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.745273 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctvdc\" (UniqueName: \"kubernetes.io/projected/02f86537-3d6d-4b62-a70f-f44d3e6f3537-kube-api-access-ctvdc\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.745506 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.745490 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/02f86537-3d6d-4b62-a70f-f44d3e6f3537-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-wm2q8\" (UID: \"02f86537-3d6d-4b62-a70f-f44d3e6f3537\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.813354 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.813317 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:30.957970 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:30.957924 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8"] Jul 22 03:03:30.962032 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:03:30.962004 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02f86537_3d6d_4b62_a70f_f44d3e6f3537.slice/crio-14508c59354a7e62fbe4018b3bad5a1d84bfe024959f81c43680834a84f69edf WatchSource:0}: Error finding container 14508c59354a7e62fbe4018b3bad5a1d84bfe024959f81c43680834a84f69edf: Status 404 returned error can't find the container with id 14508c59354a7e62fbe4018b3bad5a1d84bfe024959f81c43680834a84f69edf Jul 22 03:03:31.930278 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:31.930237 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" event={"ID":"02f86537-3d6d-4b62-a70f-f44d3e6f3537","Type":"ContainerStarted","Data":"14508c59354a7e62fbe4018b3bad5a1d84bfe024959f81c43680834a84f69edf"} Jul 22 03:03:32.522959 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.522925 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz"] Jul 22 03:03:32.526287 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.526262 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" Jul 22 03:03:32.530396 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.530370 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 22 03:03:32.536524 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.536502 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 22 03:03:32.536645 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.536586 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-f8ldd\"" Jul 22 03:03:32.536741 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.536727 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 22 03:03:32.551363 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.551334 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz"] Jul 22 03:03:32.652522 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.652490 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/36df533d-c856-4943-8b75-bb07342c17c0-manager-config\") pod \"lws-controller-manager-58c8df6b5b-vjsdz\" (UID: \"36df533d-c856-4943-8b75-bb07342c17c0\") " pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" Jul 22 03:03:32.652743 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.652548 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7jbm\" (UniqueName: \"kubernetes.io/projected/36df533d-c856-4943-8b75-bb07342c17c0-kube-api-access-t7jbm\") pod \"lws-controller-manager-58c8df6b5b-vjsdz\" (UID: \"36df533d-c856-4943-8b75-bb07342c17c0\") " pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" Jul 22 03:03:32.652743 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.652667 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36df533d-c856-4943-8b75-bb07342c17c0-cert\") pod \"lws-controller-manager-58c8df6b5b-vjsdz\" (UID: \"36df533d-c856-4943-8b75-bb07342c17c0\") " pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" Jul 22 03:03:32.652891 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.652753 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/36df533d-c856-4943-8b75-bb07342c17c0-metrics-cert\") pod \"lws-controller-manager-58c8df6b5b-vjsdz\" (UID: \"36df533d-c856-4943-8b75-bb07342c17c0\") " pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" Jul 22 03:03:32.753526 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.753482 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/36df533d-c856-4943-8b75-bb07342c17c0-manager-config\") pod \"lws-controller-manager-58c8df6b5b-vjsdz\" (UID: \"36df533d-c856-4943-8b75-bb07342c17c0\") " pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" Jul 22 03:03:32.753755 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.753537 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7jbm\" (UniqueName: \"kubernetes.io/projected/36df533d-c856-4943-8b75-bb07342c17c0-kube-api-access-t7jbm\") pod \"lws-controller-manager-58c8df6b5b-vjsdz\" (UID: \"36df533d-c856-4943-8b75-bb07342c17c0\") " pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" Jul 22 03:03:32.753755 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.753668 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36df533d-c856-4943-8b75-bb07342c17c0-cert\") pod \"lws-controller-manager-58c8df6b5b-vjsdz\" (UID: \"36df533d-c856-4943-8b75-bb07342c17c0\") " pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" Jul 22 03:03:32.753755 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.753752 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/36df533d-c856-4943-8b75-bb07342c17c0-metrics-cert\") pod \"lws-controller-manager-58c8df6b5b-vjsdz\" (UID: \"36df533d-c856-4943-8b75-bb07342c17c0\") " pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" Jul 22 03:03:32.754339 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.754310 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/36df533d-c856-4943-8b75-bb07342c17c0-manager-config\") pod \"lws-controller-manager-58c8df6b5b-vjsdz\" (UID: \"36df533d-c856-4943-8b75-bb07342c17c0\") " pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" Jul 22 03:03:32.756488 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.756464 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/36df533d-c856-4943-8b75-bb07342c17c0-metrics-cert\") pod \"lws-controller-manager-58c8df6b5b-vjsdz\" (UID: \"36df533d-c856-4943-8b75-bb07342c17c0\") " pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" Jul 22 03:03:32.756607 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.756568 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/36df533d-c856-4943-8b75-bb07342c17c0-cert\") pod \"lws-controller-manager-58c8df6b5b-vjsdz\" (UID: \"36df533d-c856-4943-8b75-bb07342c17c0\") " pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" Jul 22 03:03:32.763271 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.763233 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7jbm\" (UniqueName: \"kubernetes.io/projected/36df533d-c856-4943-8b75-bb07342c17c0-kube-api-access-t7jbm\") pod \"lws-controller-manager-58c8df6b5b-vjsdz\" (UID: \"36df533d-c856-4943-8b75-bb07342c17c0\") " pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" Jul 22 03:03:32.836776 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:32.836685 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" Jul 22 03:03:33.454674 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:33.454630 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 03:03:33.455074 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:33.454704 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 22 03:03:33.529931 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:33.529768 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz"] Jul 22 03:03:33.532433 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:03:33.532408 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36df533d_c856_4943_8b75_bb07342c17c0.slice/crio-ca2904ca53fd29ee35457e11650dccd5f7c870cd9d8484c6b823fd6e08fc346d WatchSource:0}: Error finding container ca2904ca53fd29ee35457e11650dccd5f7c870cd9d8484c6b823fd6e08fc346d: Status 404 returned error can't find the container with id ca2904ca53fd29ee35457e11650dccd5f7c870cd9d8484c6b823fd6e08fc346d Jul 22 03:03:33.953651 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:33.953616 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" event={"ID":"02f86537-3d6d-4b62-a70f-f44d3e6f3537","Type":"ContainerStarted","Data":"2ac51d30e51de530ada3d45128d71340cc3cb30915f737282b1fb223f26f409b"} Jul 22 03:03:33.953935 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:33.953724 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:33.955296 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:33.955270 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" event={"ID":"36df533d-c856-4943-8b75-bb07342c17c0","Type":"ContainerStarted","Data":"ca2904ca53fd29ee35457e11650dccd5f7c870cd9d8484c6b823fd6e08fc346d"} Jul 22 03:03:34.024761 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:34.024693 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" podStartSLOduration=1.534482545 podStartE2EDuration="4.024669534s" totalImagesPullingTime="2.490186989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:03:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:03:34.021022774 +0000 UTC m=+343.212217078" watchObservedRunningTime="2026-07-22 03:03:34.024669534 +0000 UTC m=+343.215863838" Jul 22 03:03:34.962124 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:34.962089 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-wm2q8" Jul 22 03:03:36.971415 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:36.971377 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" event={"ID":"36df533d-c856-4943-8b75-bb07342c17c0","Type":"ContainerStarted","Data":"a63beafb57d38472c5385f478f2ca12d67ae9e204597a7d158b6bc618b4dd47e"} Jul 22 03:03:36.971943 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:36.971512 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" Jul 22 03:03:47.978875 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:47.978770 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" Jul 22 03:03:48.002609 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:03:48.002555 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-58c8df6b5b-vjsdz" podStartSLOduration=13.376426536 podStartE2EDuration="16.00253768s" totalImagesPullingTime="2.626111144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:03:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:03:36.99471268 +0000 UTC m=+346.185906982" watchObservedRunningTime="2026-07-22 03:03:48.00253768 +0000 UTC m=+357.193731983" Jul 22 03:05:16.843155 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:16.843073 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd"] Jul 22 03:05:16.846235 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:16.846214 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd" Jul 22 03:05:16.850056 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:16.850035 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j5crl\"" Jul 22 03:05:16.850441 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:16.850423 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 03:05:16.850441 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:16.850435 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 03:05:16.863915 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:16.863881 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd"] Jul 22 03:05:16.958317 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:16.958274 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd"] Jul 22 03:05:16.958638 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:05:16.958612 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-zczrp], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd" podUID="7eb795a7-8dd1-4c43-832e-2f228c121abf" Jul 22 03:05:16.972601 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:16.972569 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd"] Jul 22 03:05:16.980948 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:16.980919 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7eb795a7-8dd1-4c43-832e-2f228c121abf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6dfxd\" (UID: \"7eb795a7-8dd1-4c43-832e-2f228c121abf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd" Jul 22 03:05:16.981118 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:16.980972 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zczrp\" (UniqueName: \"kubernetes.io/projected/7eb795a7-8dd1-4c43-832e-2f228c121abf-kube-api-access-zczrp\") pod \"kuadrant-operator-controller-manager-86878c769f-6dfxd\" (UID: \"7eb795a7-8dd1-4c43-832e-2f228c121abf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd" Jul 22 03:05:17.028801 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.028758 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8"] Jul 22 03:05:17.036155 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.036126 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" Jul 22 03:05:17.055240 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.055210 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8"] Jul 22 03:05:17.082212 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.082178 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zczrp\" (UniqueName: \"kubernetes.io/projected/7eb795a7-8dd1-4c43-832e-2f228c121abf-kube-api-access-zczrp\") pod \"kuadrant-operator-controller-manager-86878c769f-6dfxd\" (UID: \"7eb795a7-8dd1-4c43-832e-2f228c121abf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd" Jul 22 03:05:17.082411 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.082273 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7eb795a7-8dd1-4c43-832e-2f228c121abf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6dfxd\" (UID: \"7eb795a7-8dd1-4c43-832e-2f228c121abf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd" Jul 22 03:05:17.082608 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.082590 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7eb795a7-8dd1-4c43-832e-2f228c121abf-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-6dfxd\" (UID: \"7eb795a7-8dd1-4c43-832e-2f228c121abf\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd" Jul 22 03:05:17.086819 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:05:17.086804 2580 projected.go:196] Error preparing data for projected volume kube-api-access-zczrp for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd: failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-6dfxd" not found Jul 22 03:05:17.086903 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:05:17.086894 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7eb795a7-8dd1-4c43-832e-2f228c121abf-kube-api-access-zczrp podName:7eb795a7-8dd1-4c43-832e-2f228c121abf nodeName:}" failed. No retries permitted until 2026-07-22 03:05:17.586876729 +0000 UTC m=+446.778071016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zczrp" (UniqueName: "kubernetes.io/projected/7eb795a7-8dd1-4c43-832e-2f228c121abf-kube-api-access-zczrp") pod "kuadrant-operator-controller-manager-86878c769f-6dfxd" (UID: "7eb795a7-8dd1-4c43-832e-2f228c121abf") : failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-6dfxd" not found Jul 22 03:05:17.183046 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.183005 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d36758af-da43-4969-9202-f7ded2c00be8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dcxc8\" (UID: \"d36758af-da43-4969-9202-f7ded2c00be8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" Jul 22 03:05:17.183257 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.183072 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4kjp\" (UniqueName: \"kubernetes.io/projected/d36758af-da43-4969-9202-f7ded2c00be8-kube-api-access-w4kjp\") pod \"kuadrant-operator-controller-manager-86878c769f-dcxc8\" (UID: \"d36758af-da43-4969-9202-f7ded2c00be8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" Jul 22 03:05:17.284247 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.284196 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d36758af-da43-4969-9202-f7ded2c00be8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dcxc8\" (UID: \"d36758af-da43-4969-9202-f7ded2c00be8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" Jul 22 03:05:17.284432 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.284263 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4kjp\" (UniqueName: \"kubernetes.io/projected/d36758af-da43-4969-9202-f7ded2c00be8-kube-api-access-w4kjp\") pod \"kuadrant-operator-controller-manager-86878c769f-dcxc8\" (UID: \"d36758af-da43-4969-9202-f7ded2c00be8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" Jul 22 03:05:17.284599 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.284580 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d36758af-da43-4969-9202-f7ded2c00be8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dcxc8\" (UID: \"d36758af-da43-4969-9202-f7ded2c00be8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" Jul 22 03:05:17.296888 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.296822 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4kjp\" (UniqueName: \"kubernetes.io/projected/d36758af-da43-4969-9202-f7ded2c00be8-kube-api-access-w4kjp\") pod \"kuadrant-operator-controller-manager-86878c769f-dcxc8\" (UID: \"d36758af-da43-4969-9202-f7ded2c00be8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" Jul 22 03:05:17.346393 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.346358 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" Jul 22 03:05:17.404132 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.404097 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd" Jul 22 03:05:17.414235 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.414204 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd" Jul 22 03:05:17.416605 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:05:17.416563 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-6dfxd\" is forbidden: User \"system:node:ip-10-0-138-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-55.ec2.internal' and this object" podUID="7eb795a7-8dd1-4c43-832e-2f228c121abf" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd" Jul 22 03:05:17.486521 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.486428 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7eb795a7-8dd1-4c43-832e-2f228c121abf-extensions-socket-volume\") pod \"7eb795a7-8dd1-4c43-832e-2f228c121abf\" (UID: \"7eb795a7-8dd1-4c43-832e-2f228c121abf\") " Jul 22 03:05:17.486738 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.486661 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zczrp\" (UniqueName: \"kubernetes.io/projected/7eb795a7-8dd1-4c43-832e-2f228c121abf-kube-api-access-zczrp\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:05:17.486793 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.486736 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eb795a7-8dd1-4c43-832e-2f228c121abf-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7eb795a7-8dd1-4c43-832e-2f228c121abf" (UID: "7eb795a7-8dd1-4c43-832e-2f228c121abf"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:05:17.492403 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.492300 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8"] Jul 22 03:05:17.495539 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:05:17.495513 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd36758af_da43_4969_9202_f7ded2c00be8.slice/crio-cd89761390517bcd0f926c89edb75f083c4ecb06fac1344de69497f9cda1542c WatchSource:0}: Error finding container cd89761390517bcd0f926c89edb75f083c4ecb06fac1344de69497f9cda1542c: Status 404 returned error can't find the container with id cd89761390517bcd0f926c89edb75f083c4ecb06fac1344de69497f9cda1542c Jul 22 03:05:17.588130 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:17.588091 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7eb795a7-8dd1-4c43-832e-2f228c121abf-extensions-socket-volume\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:05:18.411908 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:18.411858 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" event={"ID":"d36758af-da43-4969-9202-f7ded2c00be8","Type":"ContainerStarted","Data":"cd89761390517bcd0f926c89edb75f083c4ecb06fac1344de69497f9cda1542c"} Jul 22 03:05:18.411908 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:18.411904 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd" Jul 22 03:05:18.414273 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:05:18.414238 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-6dfxd\" is forbidden: User \"system:node:ip-10-0-138-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-55.ec2.internal' and this object" podUID="7eb795a7-8dd1-4c43-832e-2f228c121abf" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd" Jul 22 03:05:18.421478 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:05:18.421440 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-6dfxd\" is forbidden: User \"system:node:ip-10-0-138-55.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-55.ec2.internal' and this object" podUID="7eb795a7-8dd1-4c43-832e-2f228c121abf" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-6dfxd" Jul 22 03:05:19.367889 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:19.367854 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7eb795a7-8dd1-4c43-832e-2f228c121abf" path="/var/lib/kubelet/pods/7eb795a7-8dd1-4c43-832e-2f228c121abf/volumes" Jul 22 03:05:22.433679 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:22.433638 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" event={"ID":"d36758af-da43-4969-9202-f7ded2c00be8","Type":"ContainerStarted","Data":"966816bf6fed2d02854e48c206c899ade34275124a4802edc4d0ac4e60cff175"} Jul 22 03:05:22.434088 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:22.433706 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" Jul 22 03:05:22.456957 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:22.456902 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" podStartSLOduration=2.013476211 podStartE2EDuration="6.456881571s" totalImagesPullingTime="4.44340536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:05:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:05:22.455083929 +0000 UTC m=+451.646278234" watchObservedRunningTime="2026-07-22 03:05:22.456881571 +0000 UTC m=+451.648075875" Jul 22 03:05:33.441195 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:05:33.441164 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" Jul 22 03:06:08.901151 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:08.901115 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b8848d99f-52bps"] Jul 22 03:06:08.907290 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:08.907265 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:08.915435 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:08.915415 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 03:06:08.916421 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:08.916393 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 03:06:08.916537 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:08.916464 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 03:06:08.916597 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:08.916465 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-srtb2\"" Jul 22 03:06:08.916719 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:08.916701 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 03:06:08.917009 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:08.916721 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 03:06:08.920335 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:08.920311 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b8848d99f-52bps"] Jul 22 03:06:08.920657 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:08.920636 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 03:06:09.022925 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.022882 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-console-oauth-config\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.022925 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.022921 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-oauth-serving-cert\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.023195 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.022948 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-service-ca\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.023195 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.023000 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccgwv\" (UniqueName: \"kubernetes.io/projected/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-kube-api-access-ccgwv\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.023195 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.023102 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-trusted-ca-bundle\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.023195 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.023152 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-console-config\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.023362 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.023207 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-console-serving-cert\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.123791 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.123753 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-service-ca\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.123791 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.123794 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccgwv\" (UniqueName: \"kubernetes.io/projected/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-kube-api-access-ccgwv\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.124061 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.123874 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-trusted-ca-bundle\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.124061 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.123905 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-console-config\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.124061 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.123943 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-console-serving-cert\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.124061 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.123999 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-console-oauth-config\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.124061 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.124021 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-oauth-serving-cert\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.124569 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.124536 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-service-ca\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.124690 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.124638 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-oauth-serving-cert\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.124690 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.124664 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-console-config\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.124770 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.124718 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-trusted-ca-bundle\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.126538 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.126517 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-console-oauth-config\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.126653 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.126541 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-console-serving-cert\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.132027 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.132004 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccgwv\" (UniqueName: \"kubernetes.io/projected/a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606-kube-api-access-ccgwv\") pod \"console-5b8848d99f-52bps\" (UID: \"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606\") " pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.217186 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.217095 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:09.351320 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.351293 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b8848d99f-52bps"] Jul 22 03:06:09.353520 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:06:09.353485 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7b1a3e5_9e1b_4c82_a56a_ff396c2ec606.slice/crio-0e8dc272ff1fd9049c4bac97749d9b376fb69056a862fd0b7ea395f9f48bce1e WatchSource:0}: Error finding container 0e8dc272ff1fd9049c4bac97749d9b376fb69056a862fd0b7ea395f9f48bce1e: Status 404 returned error can't find the container with id 0e8dc272ff1fd9049c4bac97749d9b376fb69056a862fd0b7ea395f9f48bce1e Jul 22 03:06:09.647669 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.647639 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b8848d99f-52bps" event={"ID":"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606","Type":"ContainerStarted","Data":"582c813297734c5a555086b36620635b446946d9a682e94f702630ecd60e5544"} Jul 22 03:06:09.647875 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.647696 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b8848d99f-52bps" event={"ID":"a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606","Type":"ContainerStarted","Data":"0e8dc272ff1fd9049c4bac97749d9b376fb69056a862fd0b7ea395f9f48bce1e"} Jul 22 03:06:09.666147 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:09.666087 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b8848d99f-52bps" podStartSLOduration=1.666072165 podStartE2EDuration="1.666072165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:06:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:06:09.664763532 +0000 UTC m=+498.855957855" watchObservedRunningTime="2026-07-22 03:06:09.666072165 +0000 UTC m=+498.857266468" Jul 22 03:06:12.189798 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:12.189768 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xspnd"] Jul 22 03:06:12.193209 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:12.193184 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xspnd" Jul 22 03:06:12.195741 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:12.195716 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 03:06:12.195880 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:12.195718 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 22 03:06:12.196536 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:12.196518 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 22 03:06:12.196633 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:12.196533 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-pjsv4\"" Jul 22 03:06:12.203662 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:12.203602 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xspnd"] Jul 22 03:06:12.252327 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:12.252288 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62szg\" (UniqueName: \"kubernetes.io/projected/2164a2af-74ed-465f-9172-0d8685e7d35f-kube-api-access-62szg\") pod \"postgres-69fb4594cf-xspnd\" (UID: \"2164a2af-74ed-465f-9172-0d8685e7d35f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xspnd" Jul 22 03:06:12.252507 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:12.252339 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2164a2af-74ed-465f-9172-0d8685e7d35f-data\") pod \"postgres-69fb4594cf-xspnd\" (UID: \"2164a2af-74ed-465f-9172-0d8685e7d35f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xspnd" Jul 22 03:06:12.353899 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:12.353861 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62szg\" (UniqueName: \"kubernetes.io/projected/2164a2af-74ed-465f-9172-0d8685e7d35f-kube-api-access-62szg\") pod \"postgres-69fb4594cf-xspnd\" (UID: \"2164a2af-74ed-465f-9172-0d8685e7d35f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xspnd" Jul 22 03:06:12.354083 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:12.353927 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2164a2af-74ed-465f-9172-0d8685e7d35f-data\") pod \"postgres-69fb4594cf-xspnd\" (UID: \"2164a2af-74ed-465f-9172-0d8685e7d35f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xspnd" Jul 22 03:06:12.354382 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:12.354363 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2164a2af-74ed-465f-9172-0d8685e7d35f-data\") pod \"postgres-69fb4594cf-xspnd\" (UID: \"2164a2af-74ed-465f-9172-0d8685e7d35f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xspnd" Jul 22 03:06:12.375584 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:12.375551 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62szg\" (UniqueName: \"kubernetes.io/projected/2164a2af-74ed-465f-9172-0d8685e7d35f-kube-api-access-62szg\") pod \"postgres-69fb4594cf-xspnd\" (UID: \"2164a2af-74ed-465f-9172-0d8685e7d35f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-xspnd" Jul 22 03:06:12.505820 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:12.505740 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xspnd" Jul 22 03:06:12.641263 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:12.641241 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-xspnd"] Jul 22 03:06:12.643685 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:06:12.643655 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2164a2af_74ed_465f_9172_0d8685e7d35f.slice/crio-0b551bb7a54131a01da52b31786fefc856d8ea5a9962a165b7aec33cb84f7549 WatchSource:0}: Error finding container 0b551bb7a54131a01da52b31786fefc856d8ea5a9962a165b7aec33cb84f7549: Status 404 returned error can't find the container with id 0b551bb7a54131a01da52b31786fefc856d8ea5a9962a165b7aec33cb84f7549 Jul 22 03:06:12.662506 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:12.662469 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xspnd" event={"ID":"2164a2af-74ed-465f-9172-0d8685e7d35f","Type":"ContainerStarted","Data":"0b551bb7a54131a01da52b31786fefc856d8ea5a9962a165b7aec33cb84f7549"} Jul 22 03:06:19.218277 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:19.218236 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:19.218776 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:19.218296 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:19.223188 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:19.223165 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:19.702736 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:19.702705 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b8848d99f-52bps" Jul 22 03:06:20.708672 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:20.708630 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xspnd" event={"ID":"2164a2af-74ed-465f-9172-0d8685e7d35f","Type":"ContainerStarted","Data":"a47058afeea1e5b505d64ea5cf953fbb7f4a3204e80a2a6991cf151adf7b6b3e"} Jul 22 03:06:20.709176 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:20.708803 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xspnd" Jul 22 03:06:20.730856 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:20.730784 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xspnd" podStartSLOduration=1.332381676 podStartE2EDuration="8.730763784s" totalImagesPullingTime="7.398382108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:06:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:06:20.729071077 +0000 UTC m=+509.920265380" watchObservedRunningTime="2026-07-22 03:06:20.730763784 +0000 UTC m=+509.921958088" Jul 22 03:06:26.742918 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:06:26.742889 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-xspnd" Jul 22 03:07:10.767058 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:10.766977 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8"] Jul 22 03:07:10.770614 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:10.770594 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8" Jul 22 03:07:10.773075 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:10.773055 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-pfdjw\"" Jul 22 03:07:10.774567 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:10.774540 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 22 03:07:10.783309 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:10.783281 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnlmx\" (UniqueName: \"kubernetes.io/projected/c3b885e7-35a8-4fe6-958b-8562b91dd8e7-kube-api-access-tnlmx\") pod \"maas-api-6f7dbbff56-d47l8\" (UID: \"c3b885e7-35a8-4fe6-958b-8562b91dd8e7\") " pod="odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8" Jul 22 03:07:10.783433 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:10.783342 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3b885e7-35a8-4fe6-958b-8562b91dd8e7-maas-api-tls\") pod \"maas-api-6f7dbbff56-d47l8\" (UID: \"c3b885e7-35a8-4fe6-958b-8562b91dd8e7\") " pod="odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8" Jul 22 03:07:10.783433 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:10.783416 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3b885e7-35a8-4fe6-958b-8562b91dd8e7-openshift-service-ca\") pod \"maas-api-6f7dbbff56-d47l8\" (UID: \"c3b885e7-35a8-4fe6-958b-8562b91dd8e7\") " pod="odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8" Jul 22 03:07:10.784879 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:10.784855 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8"] Jul 22 03:07:10.884055 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:10.884019 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3b885e7-35a8-4fe6-958b-8562b91dd8e7-openshift-service-ca\") pod \"maas-api-6f7dbbff56-d47l8\" (UID: \"c3b885e7-35a8-4fe6-958b-8562b91dd8e7\") " pod="odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8" Jul 22 03:07:10.884225 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:10.884086 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnlmx\" (UniqueName: \"kubernetes.io/projected/c3b885e7-35a8-4fe6-958b-8562b91dd8e7-kube-api-access-tnlmx\") pod \"maas-api-6f7dbbff56-d47l8\" (UID: \"c3b885e7-35a8-4fe6-958b-8562b91dd8e7\") " pod="odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8" Jul 22 03:07:10.884225 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:10.884112 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3b885e7-35a8-4fe6-958b-8562b91dd8e7-maas-api-tls\") pod \"maas-api-6f7dbbff56-d47l8\" (UID: \"c3b885e7-35a8-4fe6-958b-8562b91dd8e7\") " pod="odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8" Jul 22 03:07:10.884672 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:10.884640 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3b885e7-35a8-4fe6-958b-8562b91dd8e7-openshift-service-ca\") pod \"maas-api-6f7dbbff56-d47l8\" (UID: \"c3b885e7-35a8-4fe6-958b-8562b91dd8e7\") " pod="odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8" Jul 22 03:07:10.886547 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:10.886524 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3b885e7-35a8-4fe6-958b-8562b91dd8e7-maas-api-tls\") pod \"maas-api-6f7dbbff56-d47l8\" (UID: \"c3b885e7-35a8-4fe6-958b-8562b91dd8e7\") " pod="odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8" Jul 22 03:07:10.895209 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:10.895186 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnlmx\" (UniqueName: \"kubernetes.io/projected/c3b885e7-35a8-4fe6-958b-8562b91dd8e7-kube-api-access-tnlmx\") pod \"maas-api-6f7dbbff56-d47l8\" (UID: \"c3b885e7-35a8-4fe6-958b-8562b91dd8e7\") " pod="odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8" Jul 22 03:07:11.088114 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:11.088017 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8" Jul 22 03:07:11.229556 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:11.229532 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8"] Jul 22 03:07:11.231723 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:07:11.231691 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3b885e7_35a8_4fe6_958b_8562b91dd8e7.slice/crio-a5d57c768fb13ff4b4722451c8645f7096e36661a60c2ee98bfcb646232f176e WatchSource:0}: Error finding container a5d57c768fb13ff4b4722451c8645f7096e36661a60c2ee98bfcb646232f176e: Status 404 returned error can't find the container with id a5d57c768fb13ff4b4722451c8645f7096e36661a60c2ee98bfcb646232f176e Jul 22 03:07:11.943490 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:11.943443 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8" event={"ID":"c3b885e7-35a8-4fe6-958b-8562b91dd8e7","Type":"ContainerStarted","Data":"a5d57c768fb13ff4b4722451c8645f7096e36661a60c2ee98bfcb646232f176e"} Jul 22 03:07:13.957091 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:13.957054 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8" event={"ID":"c3b885e7-35a8-4fe6-958b-8562b91dd8e7","Type":"ContainerStarted","Data":"d53695ee9392c44b41870cd795b85cd83322597b4b835e410e372a9f1fcd4c2f"} Jul 22 03:07:13.957503 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:13.957164 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8" Jul 22 03:07:13.978078 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:13.978023 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8" podStartSLOduration=1.7487688380000002 podStartE2EDuration="3.978007448s" totalImagesPullingTime="2.22923861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:07:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:07:13.977530337 +0000 UTC m=+563.168724661" watchObservedRunningTime="2026-07-22 03:07:13.978007448 +0000 UTC m=+563.169201751" Jul 22 03:07:19.967317 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:19.967287 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6f7dbbff56-d47l8" Jul 22 03:07:51.248315 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:51.248278 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:07:51.251121 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:51.251097 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:07:51.255198 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:51.255180 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:07:51.257894 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:07:51.257876 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:08:18.301046 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.300966 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr"] Jul 22 03:08:18.304547 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.304528 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.307215 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.307193 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 03:08:18.307766 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.307750 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hkxsv\"" Jul 22 03:08:18.307824 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.307768 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 03:08:18.307880 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.307818 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 22 03:08:18.316450 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.316422 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr"] Jul 22 03:08:18.385219 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.385183 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-624ln\" (UniqueName: \"kubernetes.io/projected/a1868f38-01ed-4cb6-a787-69cfa5088f80-kube-api-access-624ln\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.385219 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.385223 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1868f38-01ed-4cb6-a787-69cfa5088f80-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.385506 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.385336 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1868f38-01ed-4cb6-a787-69cfa5088f80-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.385506 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.385371 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1868f38-01ed-4cb6-a787-69cfa5088f80-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.385506 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.385404 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1868f38-01ed-4cb6-a787-69cfa5088f80-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.385506 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.385429 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1868f38-01ed-4cb6-a787-69cfa5088f80-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.486529 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.486479 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-624ln\" (UniqueName: \"kubernetes.io/projected/a1868f38-01ed-4cb6-a787-69cfa5088f80-kube-api-access-624ln\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.486529 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.486524 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1868f38-01ed-4cb6-a787-69cfa5088f80-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.486808 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.486582 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1868f38-01ed-4cb6-a787-69cfa5088f80-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.486808 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.486599 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1868f38-01ed-4cb6-a787-69cfa5088f80-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.486985 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.486887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1868f38-01ed-4cb6-a787-69cfa5088f80-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.486985 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.486965 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1868f38-01ed-4cb6-a787-69cfa5088f80-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.487095 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.487016 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1868f38-01ed-4cb6-a787-69cfa5088f80-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.487095 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.487052 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1868f38-01ed-4cb6-a787-69cfa5088f80-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.487095 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.487062 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1868f38-01ed-4cb6-a787-69cfa5088f80-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.488999 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.488979 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1868f38-01ed-4cb6-a787-69cfa5088f80-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.489467 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.489444 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1868f38-01ed-4cb6-a787-69cfa5088f80-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.494907 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.494886 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-624ln\" (UniqueName: \"kubernetes.io/projected/a1868f38-01ed-4cb6-a787-69cfa5088f80-kube-api-access-624ln\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr\" (UID: \"a1868f38-01ed-4cb6-a787-69cfa5088f80\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.615951 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.615914 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:18.760640 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.760613 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr"] Jul 22 03:08:18.762408 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:08:18.762383 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1868f38_01ed_4cb6_a787_69cfa5088f80.slice/crio-0b2ce6b470d46f346e4de579e7839913221a70182ad05b9d836b5f7d3677352b WatchSource:0}: Error finding container 0b2ce6b470d46f346e4de579e7839913221a70182ad05b9d836b5f7d3677352b: Status 404 returned error can't find the container with id 0b2ce6b470d46f346e4de579e7839913221a70182ad05b9d836b5f7d3677352b Jul 22 03:08:18.763903 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:18.763884 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 03:08:19.256189 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:19.256152 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" event={"ID":"a1868f38-01ed-4cb6-a787-69cfa5088f80","Type":"ContainerStarted","Data":"0b2ce6b470d46f346e4de579e7839913221a70182ad05b9d836b5f7d3677352b"} Jul 22 03:08:20.262530 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:20.262492 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" event={"ID":"a1868f38-01ed-4cb6-a787-69cfa5088f80","Type":"ContainerStarted","Data":"fc2d3a89c468233326e97183245d8a1d4c3ca324f276df77572522ea6c667aa4"} Jul 22 03:08:20.286763 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:20.286697 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" podStartSLOduration=1.229859817 podStartE2EDuration="2.286676243s" totalImagesPullingTime="1.056816426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:08:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:20.285789394 +0000 UTC m=+629.476983701" watchObservedRunningTime="2026-07-22 03:08:20.286676243 +0000 UTC m=+629.477870545" Jul 22 03:08:28.616327 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:28.616282 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:28.616900 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:28.616352 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:28.629348 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:28.629318 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:29.313693 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:29.313667 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr" Jul 22 03:08:33.014022 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.013986 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l"] Jul 22 03:08:33.017900 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.017877 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.020346 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.020320 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 22 03:08:33.027324 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.027299 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l"] Jul 22 03:08:33.120790 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.120760 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v765k\" (UniqueName: \"kubernetes.io/projected/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-kube-api-access-v765k\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.120790 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.120801 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.121020 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.120914 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.121020 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.120943 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.121020 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.120968 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.121117 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.121037 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.222054 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.222012 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v765k\" (UniqueName: \"kubernetes.io/projected/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-kube-api-access-v765k\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.222262 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.222065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.222262 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.222119 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.222262 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.222134 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.222262 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.222152 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.222262 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.222210 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.222558 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.222530 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.222627 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.222598 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.222669 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.222622 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.224429 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.224400 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.224611 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.224591 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.230622 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.230597 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v765k\" (UniqueName: \"kubernetes.io/projected/4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf-kube-api-access-v765k\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rpz4l\" (UID: \"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.328591 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.328476 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:33.467343 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:33.467244 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l"] Jul 22 03:08:33.469420 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:08:33.469393 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d30d12f_3ee6_4b3d_9a5d_28efed8b5abf.slice/crio-c2558091879c3f0b1beb5bfc6633c15e1cbf13798d85e326b2ff0950279b2e22 WatchSource:0}: Error finding container c2558091879c3f0b1beb5bfc6633c15e1cbf13798d85e326b2ff0950279b2e22: Status 404 returned error can't find the container with id c2558091879c3f0b1beb5bfc6633c15e1cbf13798d85e326b2ff0950279b2e22 Jul 22 03:08:34.327643 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:34.327605 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" event={"ID":"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf","Type":"ContainerStarted","Data":"4a3f590befcf37d3a74c2bf737cc126ee646d55ca79e5032e5f35733e174d32d"} Jul 22 03:08:34.328206 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:34.327658 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" event={"ID":"4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf","Type":"ContainerStarted","Data":"c2558091879c3f0b1beb5bfc6633c15e1cbf13798d85e326b2ff0950279b2e22"} Jul 22 03:08:34.348602 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:34.348540 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" podStartSLOduration=2.121342216 podStartE2EDuration="2.348519346s" totalImagesPullingTime="227.17713ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:08:34.347100114 +0000 UTC m=+643.538294417" watchObservedRunningTime="2026-07-22 03:08:34.348519346 +0000 UTC m=+643.539713649" Jul 22 03:08:43.328621 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:43.328578 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:43.329052 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:43.328665 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:43.340458 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:43.340433 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:08:43.384442 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:08:43.384414 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rpz4l" Jul 22 03:12:51.285139 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:12:51.285060 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:12:51.285615 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:12:51.285405 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:12:51.291947 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:12:51.291925 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:12:51.292225 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:12:51.292209 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:16:32.518261 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:32.518219 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8"] Jul 22 03:16:32.518884 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:32.518436 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" podUID="d36758af-da43-4969-9202-f7ded2c00be8" containerName="manager" containerID="cri-o://966816bf6fed2d02854e48c206c899ade34275124a4802edc4d0ac4e60cff175" gracePeriod=10 Jul 22 03:16:35.981366 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:35.981297 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" Jul 22 03:16:36.033887 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:36.033854 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d36758af-da43-4969-9202-f7ded2c00be8-extensions-socket-volume\") pod \"d36758af-da43-4969-9202-f7ded2c00be8\" (UID: \"d36758af-da43-4969-9202-f7ded2c00be8\") " Jul 22 03:16:36.034049 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:36.033934 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4kjp\" (UniqueName: \"kubernetes.io/projected/d36758af-da43-4969-9202-f7ded2c00be8-kube-api-access-w4kjp\") pod \"d36758af-da43-4969-9202-f7ded2c00be8\" (UID: \"d36758af-da43-4969-9202-f7ded2c00be8\") " Jul 22 03:16:36.034252 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:36.034227 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d36758af-da43-4969-9202-f7ded2c00be8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d36758af-da43-4969-9202-f7ded2c00be8" (UID: "d36758af-da43-4969-9202-f7ded2c00be8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:16:36.034304 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:36.034291 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d36758af-da43-4969-9202-f7ded2c00be8-extensions-socket-volume\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:16:36.036024 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:36.036000 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d36758af-da43-4969-9202-f7ded2c00be8-kube-api-access-w4kjp" (OuterVolumeSpecName: "kube-api-access-w4kjp") pod "d36758af-da43-4969-9202-f7ded2c00be8" (UID: "d36758af-da43-4969-9202-f7ded2c00be8"). InnerVolumeSpecName "kube-api-access-w4kjp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:16:36.135232 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:36.135189 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4kjp\" (UniqueName: \"kubernetes.io/projected/d36758af-da43-4969-9202-f7ded2c00be8-kube-api-access-w4kjp\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:16:36.614150 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:36.614113 2580 generic.go:356] "Generic (PLEG): container finished" podID="d36758af-da43-4969-9202-f7ded2c00be8" containerID="966816bf6fed2d02854e48c206c899ade34275124a4802edc4d0ac4e60cff175" exitCode=0 Jul 22 03:16:36.614348 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:36.614168 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" Jul 22 03:16:36.614348 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:36.614205 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" event={"ID":"d36758af-da43-4969-9202-f7ded2c00be8","Type":"ContainerDied","Data":"966816bf6fed2d02854e48c206c899ade34275124a4802edc4d0ac4e60cff175"} Jul 22 03:16:36.614348 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:36.614258 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8" event={"ID":"d36758af-da43-4969-9202-f7ded2c00be8","Type":"ContainerDied","Data":"cd89761390517bcd0f926c89edb75f083c4ecb06fac1344de69497f9cda1542c"} Jul 22 03:16:36.614348 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:36.614280 2580 scope.go:117] "RemoveContainer" containerID="966816bf6fed2d02854e48c206c899ade34275124a4802edc4d0ac4e60cff175" Jul 22 03:16:36.624142 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:36.624115 2580 scope.go:117] "RemoveContainer" containerID="966816bf6fed2d02854e48c206c899ade34275124a4802edc4d0ac4e60cff175" Jul 22 03:16:36.624455 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:16:36.624436 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"966816bf6fed2d02854e48c206c899ade34275124a4802edc4d0ac4e60cff175\": container with ID starting with 966816bf6fed2d02854e48c206c899ade34275124a4802edc4d0ac4e60cff175 not found: ID does not exist" containerID="966816bf6fed2d02854e48c206c899ade34275124a4802edc4d0ac4e60cff175" Jul 22 03:16:36.624535 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:36.624461 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"966816bf6fed2d02854e48c206c899ade34275124a4802edc4d0ac4e60cff175"} err="failed to get container status \"966816bf6fed2d02854e48c206c899ade34275124a4802edc4d0ac4e60cff175\": rpc error: code = NotFound desc = could not find container \"966816bf6fed2d02854e48c206c899ade34275124a4802edc4d0ac4e60cff175\": container with ID starting with 966816bf6fed2d02854e48c206c899ade34275124a4802edc4d0ac4e60cff175 not found: ID does not exist" Jul 22 03:16:36.636382 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:36.636349 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8"] Jul 22 03:16:36.640380 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:36.640352 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dcxc8"] Jul 22 03:16:37.366937 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:16:37.366902 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d36758af-da43-4969-9202-f7ded2c00be8" path="/var/lib/kubelet/pods/d36758af-da43-4969-9202-f7ded2c00be8/volumes" Jul 22 03:17:33.751266 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:33.751180 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r99n7"] Jul 22 03:17:33.751730 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:33.751557 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d36758af-da43-4969-9202-f7ded2c00be8" containerName="manager" Jul 22 03:17:33.751730 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:33.751568 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d36758af-da43-4969-9202-f7ded2c00be8" containerName="manager" Jul 22 03:17:33.751730 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:33.751626 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d36758af-da43-4969-9202-f7ded2c00be8" containerName="manager" Jul 22 03:17:33.754809 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:33.754788 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r99n7" Jul 22 03:17:33.757990 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:33.757971 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 03:17:33.758119 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:33.758027 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 03:17:33.758119 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:33.758035 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j5crl\"" Jul 22 03:17:33.769921 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:33.769890 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r99n7"] Jul 22 03:17:33.859523 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:33.859494 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8c6f392-6a4f-4774-8908-6c54cd7130cd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-r99n7\" (UID: \"e8c6f392-6a4f-4774-8908-6c54cd7130cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r99n7" Jul 22 03:17:33.859694 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:33.859530 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjr4d\" (UniqueName: \"kubernetes.io/projected/e8c6f392-6a4f-4774-8908-6c54cd7130cd-kube-api-access-mjr4d\") pod \"kuadrant-operator-controller-manager-86878c769f-r99n7\" (UID: \"e8c6f392-6a4f-4774-8908-6c54cd7130cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r99n7" Jul 22 03:17:33.961075 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:33.961035 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8c6f392-6a4f-4774-8908-6c54cd7130cd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-r99n7\" (UID: \"e8c6f392-6a4f-4774-8908-6c54cd7130cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r99n7" Jul 22 03:17:33.961075 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:33.961074 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjr4d\" (UniqueName: \"kubernetes.io/projected/e8c6f392-6a4f-4774-8908-6c54cd7130cd-kube-api-access-mjr4d\") pod \"kuadrant-operator-controller-manager-86878c769f-r99n7\" (UID: \"e8c6f392-6a4f-4774-8908-6c54cd7130cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r99n7" Jul 22 03:17:33.961430 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:33.961409 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8c6f392-6a4f-4774-8908-6c54cd7130cd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-r99n7\" (UID: \"e8c6f392-6a4f-4774-8908-6c54cd7130cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r99n7" Jul 22 03:17:33.969794 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:33.969758 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjr4d\" (UniqueName: \"kubernetes.io/projected/e8c6f392-6a4f-4774-8908-6c54cd7130cd-kube-api-access-mjr4d\") pod \"kuadrant-operator-controller-manager-86878c769f-r99n7\" (UID: \"e8c6f392-6a4f-4774-8908-6c54cd7130cd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r99n7" Jul 22 03:17:34.065019 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:34.064927 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r99n7" Jul 22 03:17:34.202885 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:34.202859 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r99n7"] Jul 22 03:17:34.204974 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:17:34.204948 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c6f392_6a4f_4774_8908_6c54cd7130cd.slice/crio-22a7cecaa52cefbc8cad4d315259937412c3b1498d2eca0479b838dfa8dca424 WatchSource:0}: Error finding container 22a7cecaa52cefbc8cad4d315259937412c3b1498d2eca0479b838dfa8dca424: Status 404 returned error can't find the container with id 22a7cecaa52cefbc8cad4d315259937412c3b1498d2eca0479b838dfa8dca424 Jul 22 03:17:34.883353 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:34.883309 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r99n7" event={"ID":"e8c6f392-6a4f-4774-8908-6c54cd7130cd","Type":"ContainerStarted","Data":"2e7d56046a083befdf8249c97f060f8188e70874d3661e4af43d79b93f04179d"} Jul 22 03:17:34.883764 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:34.883406 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r99n7" Jul 22 03:17:34.883764 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:34.883424 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r99n7" event={"ID":"e8c6f392-6a4f-4774-8908-6c54cd7130cd","Type":"ContainerStarted","Data":"22a7cecaa52cefbc8cad4d315259937412c3b1498d2eca0479b838dfa8dca424"} Jul 22 03:17:34.904257 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:34.904186 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r99n7" podStartSLOduration=1.904170724 podStartE2EDuration="1.904170724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:17:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:17:34.90326412 +0000 UTC m=+1184.094458443" watchObservedRunningTime="2026-07-22 03:17:34.904170724 +0000 UTC m=+1184.095365027" Jul 22 03:17:45.890363 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:45.890328 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-r99n7" Jul 22 03:17:51.312757 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:51.312724 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:17:51.315753 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:51.315723 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:17:51.320960 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:51.320935 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:17:51.323290 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:17:51.323256 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:22:51.345485 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:22:51.345453 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:22:51.348469 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:22:51.348446 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:22:51.352162 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:22:51.352136 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:22:51.355027 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:22:51.355010 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:24:45.616510 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:45.616410 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6"] Jul 22 03:24:45.620330 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:45.620306 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" Jul 22 03:24:45.622600 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:45.622578 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0c81ed29-serving-cert\"" Jul 22 03:24:45.629090 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:45.629064 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6"] Jul 22 03:24:45.787233 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:45.787198 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrd2h\" (UniqueName: \"kubernetes.io/projected/fda00a9f-4f67-4427-a9dd-f3b81011d094-kube-api-access-rrd2h\") pod \"maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6\" (UID: \"fda00a9f-4f67-4427-a9dd-f3b81011d094\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" Jul 22 03:24:45.787406 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:45.787257 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fda00a9f-4f67-4427-a9dd-f3b81011d094-openshift-service-ca\") pod \"maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6\" (UID: \"fda00a9f-4f67-4427-a9dd-f3b81011d094\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" Jul 22 03:24:45.787406 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:45.787355 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fda00a9f-4f67-4427-a9dd-f3b81011d094-maas-api-tls\") pod \"maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6\" (UID: \"fda00a9f-4f67-4427-a9dd-f3b81011d094\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" Jul 22 03:24:45.888056 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:45.887974 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fda00a9f-4f67-4427-a9dd-f3b81011d094-maas-api-tls\") pod \"maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6\" (UID: \"fda00a9f-4f67-4427-a9dd-f3b81011d094\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" Jul 22 03:24:45.888056 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:45.888030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrd2h\" (UniqueName: \"kubernetes.io/projected/fda00a9f-4f67-4427-a9dd-f3b81011d094-kube-api-access-rrd2h\") pod \"maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6\" (UID: \"fda00a9f-4f67-4427-a9dd-f3b81011d094\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" Jul 22 03:24:45.888253 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:45.888063 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fda00a9f-4f67-4427-a9dd-f3b81011d094-openshift-service-ca\") pod \"maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6\" (UID: \"fda00a9f-4f67-4427-a9dd-f3b81011d094\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" Jul 22 03:24:45.888719 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:45.888690 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fda00a9f-4f67-4427-a9dd-f3b81011d094-openshift-service-ca\") pod \"maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6\" (UID: \"fda00a9f-4f67-4427-a9dd-f3b81011d094\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" Jul 22 03:24:45.890492 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:45.890471 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fda00a9f-4f67-4427-a9dd-f3b81011d094-maas-api-tls\") pod \"maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6\" (UID: \"fda00a9f-4f67-4427-a9dd-f3b81011d094\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" Jul 22 03:24:45.898199 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:45.898177 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrd2h\" (UniqueName: \"kubernetes.io/projected/fda00a9f-4f67-4427-a9dd-f3b81011d094-kube-api-access-rrd2h\") pod \"maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6\" (UID: \"fda00a9f-4f67-4427-a9dd-f3b81011d094\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" Jul 22 03:24:45.932190 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:45.932166 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" Jul 22 03:24:46.276662 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:46.276636 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6"] Jul 22 03:24:46.279220 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:24:46.279187 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfda00a9f_4f67_4427_a9dd_f3b81011d094.slice/crio-26d3e8e115240d2472a2ef88fed2d53fbd8e575677908f2a9b11003b802c8e21 WatchSource:0}: Error finding container 26d3e8e115240d2472a2ef88fed2d53fbd8e575677908f2a9b11003b802c8e21: Status 404 returned error can't find the container with id 26d3e8e115240d2472a2ef88fed2d53fbd8e575677908f2a9b11003b802c8e21 Jul 22 03:24:46.280621 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:46.280602 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 03:24:46.925227 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:46.925184 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" event={"ID":"fda00a9f-4f67-4427-a9dd-f3b81011d094","Type":"ContainerStarted","Data":"f9f659e8d002a46a05cb8cfed3c2d63bf5c56273d439b9d7aa5d448932ae6117"} Jul 22 03:24:46.925620 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:46.925254 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" event={"ID":"fda00a9f-4f67-4427-a9dd-f3b81011d094","Type":"ContainerStarted","Data":"26d3e8e115240d2472a2ef88fed2d53fbd8e575677908f2a9b11003b802c8e21"} Jul 22 03:24:46.925620 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:46.925316 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" Jul 22 03:24:46.946065 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:46.946009 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" podStartSLOduration=1.4156896319999999 podStartE2EDuration="1.945994114s" totalImagesPullingTime="530.304482ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:24:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:24:46.94467717 +0000 UTC m=+1616.135871477" watchObservedRunningTime="2026-07-22 03:24:46.945994114 +0000 UTC m=+1616.137188416" Jul 22 03:24:52.935360 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:24:52.935329 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" Jul 22 03:25:21.749773 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:21.749736 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6"] Jul 22 03:25:21.750408 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:21.749976 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" podUID="fda00a9f-4f67-4427-a9dd-f3b81011d094" containerName="maas-api" containerID="cri-o://f9f659e8d002a46a05cb8cfed3c2d63bf5c56273d439b9d7aa5d448932ae6117" gracePeriod=30 Jul 22 03:25:22.003051 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.002982 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" Jul 22 03:25:22.092532 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.092493 2580 generic.go:356] "Generic (PLEG): container finished" podID="fda00a9f-4f67-4427-a9dd-f3b81011d094" containerID="f9f659e8d002a46a05cb8cfed3c2d63bf5c56273d439b9d7aa5d448932ae6117" exitCode=0 Jul 22 03:25:22.092735 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.092545 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" Jul 22 03:25:22.092735 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.092581 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" event={"ID":"fda00a9f-4f67-4427-a9dd-f3b81011d094","Type":"ContainerDied","Data":"f9f659e8d002a46a05cb8cfed3c2d63bf5c56273d439b9d7aa5d448932ae6117"} Jul 22 03:25:22.092735 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.092636 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6" event={"ID":"fda00a9f-4f67-4427-a9dd-f3b81011d094","Type":"ContainerDied","Data":"26d3e8e115240d2472a2ef88fed2d53fbd8e575677908f2a9b11003b802c8e21"} Jul 22 03:25:22.092735 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.092661 2580 scope.go:117] "RemoveContainer" containerID="f9f659e8d002a46a05cb8cfed3c2d63bf5c56273d439b9d7aa5d448932ae6117" Jul 22 03:25:22.101015 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.100997 2580 scope.go:117] "RemoveContainer" containerID="f9f659e8d002a46a05cb8cfed3c2d63bf5c56273d439b9d7aa5d448932ae6117" Jul 22 03:25:22.101290 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:25:22.101270 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9f659e8d002a46a05cb8cfed3c2d63bf5c56273d439b9d7aa5d448932ae6117\": container with ID starting with f9f659e8d002a46a05cb8cfed3c2d63bf5c56273d439b9d7aa5d448932ae6117 not found: ID does not exist" containerID="f9f659e8d002a46a05cb8cfed3c2d63bf5c56273d439b9d7aa5d448932ae6117" Jul 22 03:25:22.101341 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.101299 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9f659e8d002a46a05cb8cfed3c2d63bf5c56273d439b9d7aa5d448932ae6117"} err="failed to get container status \"f9f659e8d002a46a05cb8cfed3c2d63bf5c56273d439b9d7aa5d448932ae6117\": rpc error: code = NotFound desc = could not find container \"f9f659e8d002a46a05cb8cfed3c2d63bf5c56273d439b9d7aa5d448932ae6117\": container with ID starting with f9f659e8d002a46a05cb8cfed3c2d63bf5c56273d439b9d7aa5d448932ae6117 not found: ID does not exist" Jul 22 03:25:22.123816 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.123785 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrd2h\" (UniqueName: \"kubernetes.io/projected/fda00a9f-4f67-4427-a9dd-f3b81011d094-kube-api-access-rrd2h\") pod \"fda00a9f-4f67-4427-a9dd-f3b81011d094\" (UID: \"fda00a9f-4f67-4427-a9dd-f3b81011d094\") " Jul 22 03:25:22.123966 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.123848 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fda00a9f-4f67-4427-a9dd-f3b81011d094-openshift-service-ca\") pod \"fda00a9f-4f67-4427-a9dd-f3b81011d094\" (UID: \"fda00a9f-4f67-4427-a9dd-f3b81011d094\") " Jul 22 03:25:22.123966 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.123898 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fda00a9f-4f67-4427-a9dd-f3b81011d094-maas-api-tls\") pod \"fda00a9f-4f67-4427-a9dd-f3b81011d094\" (UID: \"fda00a9f-4f67-4427-a9dd-f3b81011d094\") " Jul 22 03:25:22.124395 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.124370 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fda00a9f-4f67-4427-a9dd-f3b81011d094-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fda00a9f-4f67-4427-a9dd-f3b81011d094" (UID: "fda00a9f-4f67-4427-a9dd-f3b81011d094"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:25:22.126116 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.126083 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fda00a9f-4f67-4427-a9dd-f3b81011d094-kube-api-access-rrd2h" (OuterVolumeSpecName: "kube-api-access-rrd2h") pod "fda00a9f-4f67-4427-a9dd-f3b81011d094" (UID: "fda00a9f-4f67-4427-a9dd-f3b81011d094"). InnerVolumeSpecName "kube-api-access-rrd2h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:25:22.126204 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.126143 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fda00a9f-4f67-4427-a9dd-f3b81011d094-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fda00a9f-4f67-4427-a9dd-f3b81011d094" (UID: "fda00a9f-4f67-4427-a9dd-f3b81011d094"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:25:22.225438 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.225395 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrd2h\" (UniqueName: \"kubernetes.io/projected/fda00a9f-4f67-4427-a9dd-f3b81011d094-kube-api-access-rrd2h\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:25:22.225438 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.225429 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fda00a9f-4f67-4427-a9dd-f3b81011d094-openshift-service-ca\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:25:22.225438 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.225441 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fda00a9f-4f67-4427-a9dd-f3b81011d094-maas-api-tls\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:25:22.414387 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.414351 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6"] Jul 22 03:25:22.418560 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:22.418533 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0c81ed29-6b887bc977-5jxl6"] Jul 22 03:25:23.367927 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:23.367895 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fda00a9f-4f67-4427-a9dd-f3b81011d094" path="/var/lib/kubelet/pods/fda00a9f-4f67-4427-a9dd-f3b81011d094/volumes" Jul 22 03:25:29.845990 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:29.845955 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp"] Jul 22 03:25:29.848476 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:29.846352 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fda00a9f-4f67-4427-a9dd-f3b81011d094" containerName="maas-api" Jul 22 03:25:29.848476 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:29.846365 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fda00a9f-4f67-4427-a9dd-f3b81011d094" containerName="maas-api" Jul 22 03:25:29.848476 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:29.846453 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fda00a9f-4f67-4427-a9dd-f3b81011d094" containerName="maas-api" Jul 22 03:25:29.849330 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:29.849311 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" Jul 22 03:25:29.851588 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:29.851559 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-321cbb2c-serving-cert\"" Jul 22 03:25:29.860958 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:29.860928 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp"] Jul 22 03:25:30.006099 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:30.006058 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/813f1329-8cc9-4b92-bad7-802472dcf701-openshift-service-ca\") pod \"maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp\" (UID: \"813f1329-8cc9-4b92-bad7-802472dcf701\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" Jul 22 03:25:30.006289 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:30.006106 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt22v\" (UniqueName: \"kubernetes.io/projected/813f1329-8cc9-4b92-bad7-802472dcf701-kube-api-access-mt22v\") pod \"maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp\" (UID: \"813f1329-8cc9-4b92-bad7-802472dcf701\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" Jul 22 03:25:30.006289 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:30.006217 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/813f1329-8cc9-4b92-bad7-802472dcf701-maas-api-tls\") pod \"maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp\" (UID: \"813f1329-8cc9-4b92-bad7-802472dcf701\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" Jul 22 03:25:30.107423 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:30.107382 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/813f1329-8cc9-4b92-bad7-802472dcf701-openshift-service-ca\") pod \"maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp\" (UID: \"813f1329-8cc9-4b92-bad7-802472dcf701\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" Jul 22 03:25:30.107613 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:30.107437 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt22v\" (UniqueName: \"kubernetes.io/projected/813f1329-8cc9-4b92-bad7-802472dcf701-kube-api-access-mt22v\") pod \"maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp\" (UID: \"813f1329-8cc9-4b92-bad7-802472dcf701\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" Jul 22 03:25:30.107613 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:30.107523 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/813f1329-8cc9-4b92-bad7-802472dcf701-maas-api-tls\") pod \"maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp\" (UID: \"813f1329-8cc9-4b92-bad7-802472dcf701\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" Jul 22 03:25:30.107733 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:25:30.107638 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-serving-cert: secret "maas-api-e2e-migrate-321cbb2c-serving-cert" not found Jul 22 03:25:30.107733 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:25:30.107730 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/813f1329-8cc9-4b92-bad7-802472dcf701-maas-api-tls podName:813f1329-8cc9-4b92-bad7-802472dcf701 nodeName:}" failed. No retries permitted until 2026-07-22 03:25:30.607706828 +0000 UTC m=+1659.798901112 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/813f1329-8cc9-4b92-bad7-802472dcf701-maas-api-tls") pod "maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" (UID: "813f1329-8cc9-4b92-bad7-802472dcf701") : secret "maas-api-e2e-migrate-321cbb2c-serving-cert" not found Jul 22 03:25:30.108111 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:30.108090 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/813f1329-8cc9-4b92-bad7-802472dcf701-openshift-service-ca\") pod \"maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp\" (UID: \"813f1329-8cc9-4b92-bad7-802472dcf701\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" Jul 22 03:25:30.153973 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:30.153939 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt22v\" (UniqueName: \"kubernetes.io/projected/813f1329-8cc9-4b92-bad7-802472dcf701-kube-api-access-mt22v\") pod \"maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp\" (UID: \"813f1329-8cc9-4b92-bad7-802472dcf701\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" Jul 22 03:25:30.611644 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:30.611614 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/813f1329-8cc9-4b92-bad7-802472dcf701-maas-api-tls\") pod \"maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp\" (UID: \"813f1329-8cc9-4b92-bad7-802472dcf701\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" Jul 22 03:25:30.614078 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:30.614051 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/813f1329-8cc9-4b92-bad7-802472dcf701-maas-api-tls\") pod \"maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp\" (UID: \"813f1329-8cc9-4b92-bad7-802472dcf701\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" Jul 22 03:25:30.762791 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:30.762754 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" Jul 22 03:25:30.894368 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:30.894295 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp"] Jul 22 03:25:30.897629 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:25:30.897604 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod813f1329_8cc9_4b92_bad7_802472dcf701.slice/crio-28b7348a905b6df07e2b4b34b5c05cb6fa4875dcc74b1f342e8c6eaf40b084ca WatchSource:0}: Error finding container 28b7348a905b6df07e2b4b34b5c05cb6fa4875dcc74b1f342e8c6eaf40b084ca: Status 404 returned error can't find the container with id 28b7348a905b6df07e2b4b34b5c05cb6fa4875dcc74b1f342e8c6eaf40b084ca Jul 22 03:25:31.136118 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:31.136080 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" event={"ID":"813f1329-8cc9-4b92-bad7-802472dcf701","Type":"ContainerStarted","Data":"28b7348a905b6df07e2b4b34b5c05cb6fa4875dcc74b1f342e8c6eaf40b084ca"} Jul 22 03:25:32.142374 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:32.142336 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" event={"ID":"813f1329-8cc9-4b92-bad7-802472dcf701","Type":"ContainerStarted","Data":"a01b774ff75a6e6d19db99b07cccad51a8a6b89dc6cb5abb9b93875e0109969b"} Jul 22 03:25:32.142861 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:32.142447 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" Jul 22 03:25:32.159646 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:32.159598 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" podStartSLOduration=2.844408588 podStartE2EDuration="3.159583709s" totalImagesPullingTime="315.175121ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:25:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:25:32.158522113 +0000 UTC m=+1661.349716439" watchObservedRunningTime="2026-07-22 03:25:32.159583709 +0000 UTC m=+1661.350778012" Jul 22 03:25:38.153037 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:25:38.153009 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" Jul 22 03:26:05.506469 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:05.506433 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp"] Jul 22 03:26:05.507000 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:05.506696 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" podUID="813f1329-8cc9-4b92-bad7-802472dcf701" containerName="maas-api" containerID="cri-o://a01b774ff75a6e6d19db99b07cccad51a8a6b89dc6cb5abb9b93875e0109969b" gracePeriod=30 Jul 22 03:26:05.756791 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:05.756727 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" Jul 22 03:26:05.837341 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:05.837305 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/813f1329-8cc9-4b92-bad7-802472dcf701-openshift-service-ca\") pod \"813f1329-8cc9-4b92-bad7-802472dcf701\" (UID: \"813f1329-8cc9-4b92-bad7-802472dcf701\") " Jul 22 03:26:05.837514 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:05.837374 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/813f1329-8cc9-4b92-bad7-802472dcf701-maas-api-tls\") pod \"813f1329-8cc9-4b92-bad7-802472dcf701\" (UID: \"813f1329-8cc9-4b92-bad7-802472dcf701\") " Jul 22 03:26:05.837514 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:05.837406 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mt22v\" (UniqueName: \"kubernetes.io/projected/813f1329-8cc9-4b92-bad7-802472dcf701-kube-api-access-mt22v\") pod \"813f1329-8cc9-4b92-bad7-802472dcf701\" (UID: \"813f1329-8cc9-4b92-bad7-802472dcf701\") " Jul 22 03:26:05.837674 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:05.837648 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/813f1329-8cc9-4b92-bad7-802472dcf701-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "813f1329-8cc9-4b92-bad7-802472dcf701" (UID: "813f1329-8cc9-4b92-bad7-802472dcf701"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:26:05.839502 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:05.839472 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/813f1329-8cc9-4b92-bad7-802472dcf701-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "813f1329-8cc9-4b92-bad7-802472dcf701" (UID: "813f1329-8cc9-4b92-bad7-802472dcf701"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:26:05.839615 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:05.839482 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/813f1329-8cc9-4b92-bad7-802472dcf701-kube-api-access-mt22v" (OuterVolumeSpecName: "kube-api-access-mt22v") pod "813f1329-8cc9-4b92-bad7-802472dcf701" (UID: "813f1329-8cc9-4b92-bad7-802472dcf701"). InnerVolumeSpecName "kube-api-access-mt22v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:26:05.938980 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:05.938948 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/813f1329-8cc9-4b92-bad7-802472dcf701-maas-api-tls\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:26:05.938980 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:05.938975 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mt22v\" (UniqueName: \"kubernetes.io/projected/813f1329-8cc9-4b92-bad7-802472dcf701-kube-api-access-mt22v\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:26:05.938980 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:05.938985 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/813f1329-8cc9-4b92-bad7-802472dcf701-openshift-service-ca\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:26:06.302629 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:06.302590 2580 generic.go:356] "Generic (PLEG): container finished" podID="813f1329-8cc9-4b92-bad7-802472dcf701" containerID="a01b774ff75a6e6d19db99b07cccad51a8a6b89dc6cb5abb9b93875e0109969b" exitCode=0 Jul 22 03:26:06.302842 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:06.302632 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" Jul 22 03:26:06.302842 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:06.302652 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" event={"ID":"813f1329-8cc9-4b92-bad7-802472dcf701","Type":"ContainerDied","Data":"a01b774ff75a6e6d19db99b07cccad51a8a6b89dc6cb5abb9b93875e0109969b"} Jul 22 03:26:06.302842 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:06.302701 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp" event={"ID":"813f1329-8cc9-4b92-bad7-802472dcf701","Type":"ContainerDied","Data":"28b7348a905b6df07e2b4b34b5c05cb6fa4875dcc74b1f342e8c6eaf40b084ca"} Jul 22 03:26:06.302842 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:06.302726 2580 scope.go:117] "RemoveContainer" containerID="a01b774ff75a6e6d19db99b07cccad51a8a6b89dc6cb5abb9b93875e0109969b" Jul 22 03:26:06.312082 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:06.312063 2580 scope.go:117] "RemoveContainer" containerID="a01b774ff75a6e6d19db99b07cccad51a8a6b89dc6cb5abb9b93875e0109969b" Jul 22 03:26:06.312360 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:26:06.312342 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a01b774ff75a6e6d19db99b07cccad51a8a6b89dc6cb5abb9b93875e0109969b\": container with ID starting with a01b774ff75a6e6d19db99b07cccad51a8a6b89dc6cb5abb9b93875e0109969b not found: ID does not exist" containerID="a01b774ff75a6e6d19db99b07cccad51a8a6b89dc6cb5abb9b93875e0109969b" Jul 22 03:26:06.312408 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:06.312369 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a01b774ff75a6e6d19db99b07cccad51a8a6b89dc6cb5abb9b93875e0109969b"} err="failed to get container status \"a01b774ff75a6e6d19db99b07cccad51a8a6b89dc6cb5abb9b93875e0109969b\": rpc error: code = NotFound desc = could not find container \"a01b774ff75a6e6d19db99b07cccad51a8a6b89dc6cb5abb9b93875e0109969b\": container with ID starting with a01b774ff75a6e6d19db99b07cccad51a8a6b89dc6cb5abb9b93875e0109969b not found: ID does not exist" Jul 22 03:26:06.325708 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:06.325679 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp"] Jul 22 03:26:06.331000 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:06.330973 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-321cbb2c-97ccbcf7b-778cp"] Jul 22 03:26:07.366967 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:07.366930 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="813f1329-8cc9-4b92-bad7-802472dcf701" path="/var/lib/kubelet/pods/813f1329-8cc9-4b92-bad7-802472dcf701/volumes" Jul 22 03:26:12.989257 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:12.989224 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww"] Jul 22 03:26:12.989628 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:12.989577 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="813f1329-8cc9-4b92-bad7-802472dcf701" containerName="maas-api" Jul 22 03:26:12.989628 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:12.989590 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="813f1329-8cc9-4b92-bad7-802472dcf701" containerName="maas-api" Jul 22 03:26:12.989700 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:12.989663 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="813f1329-8cc9-4b92-bad7-802472dcf701" containerName="maas-api" Jul 22 03:26:12.993860 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:12.993839 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" Jul 22 03:26:12.998315 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:12.998297 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-67a1aa21-serving-cert\"" Jul 22 03:26:13.021771 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:13.021743 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww"] Jul 22 03:26:13.105312 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:13.105278 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb56k\" (UniqueName: \"kubernetes.io/projected/d14af780-53d7-4fe5-8d63-3324458dc4ca-kube-api-access-jb56k\") pod \"maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww\" (UID: \"d14af780-53d7-4fe5-8d63-3324458dc4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" Jul 22 03:26:13.105487 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:13.105375 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d14af780-53d7-4fe5-8d63-3324458dc4ca-maas-api-tls\") pod \"maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww\" (UID: \"d14af780-53d7-4fe5-8d63-3324458dc4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" Jul 22 03:26:13.105487 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:13.105401 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d14af780-53d7-4fe5-8d63-3324458dc4ca-openshift-service-ca\") pod \"maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww\" (UID: \"d14af780-53d7-4fe5-8d63-3324458dc4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" Jul 22 03:26:13.206502 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:13.206464 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb56k\" (UniqueName: \"kubernetes.io/projected/d14af780-53d7-4fe5-8d63-3324458dc4ca-kube-api-access-jb56k\") pod \"maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww\" (UID: \"d14af780-53d7-4fe5-8d63-3324458dc4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" Jul 22 03:26:13.206694 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:13.206537 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d14af780-53d7-4fe5-8d63-3324458dc4ca-maas-api-tls\") pod \"maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww\" (UID: \"d14af780-53d7-4fe5-8d63-3324458dc4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" Jul 22 03:26:13.206694 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:13.206561 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d14af780-53d7-4fe5-8d63-3324458dc4ca-openshift-service-ca\") pod \"maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww\" (UID: \"d14af780-53d7-4fe5-8d63-3324458dc4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" Jul 22 03:26:13.206822 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:26:13.206689 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-serving-cert: secret "maas-api-e2e-ait-67a1aa21-serving-cert" not found Jul 22 03:26:13.206822 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:26:13.206782 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d14af780-53d7-4fe5-8d63-3324458dc4ca-maas-api-tls podName:d14af780-53d7-4fe5-8d63-3324458dc4ca nodeName:}" failed. No retries permitted until 2026-07-22 03:26:13.706758501 +0000 UTC m=+1702.897952796 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d14af780-53d7-4fe5-8d63-3324458dc4ca-maas-api-tls") pod "maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" (UID: "d14af780-53d7-4fe5-8d63-3324458dc4ca") : secret "maas-api-e2e-ait-67a1aa21-serving-cert" not found Jul 22 03:26:13.207224 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:13.207205 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d14af780-53d7-4fe5-8d63-3324458dc4ca-openshift-service-ca\") pod \"maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww\" (UID: \"d14af780-53d7-4fe5-8d63-3324458dc4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" Jul 22 03:26:13.215697 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:13.215667 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb56k\" (UniqueName: \"kubernetes.io/projected/d14af780-53d7-4fe5-8d63-3324458dc4ca-kube-api-access-jb56k\") pod \"maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww\" (UID: \"d14af780-53d7-4fe5-8d63-3324458dc4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" Jul 22 03:26:13.711979 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:13.711937 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d14af780-53d7-4fe5-8d63-3324458dc4ca-maas-api-tls\") pod \"maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww\" (UID: \"d14af780-53d7-4fe5-8d63-3324458dc4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" Jul 22 03:26:13.714588 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:13.714552 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d14af780-53d7-4fe5-8d63-3324458dc4ca-maas-api-tls\") pod \"maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww\" (UID: \"d14af780-53d7-4fe5-8d63-3324458dc4ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" Jul 22 03:26:13.903742 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:13.903706 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" Jul 22 03:26:14.043103 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:14.043077 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww"] Jul 22 03:26:14.045211 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:26:14.045189 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd14af780_53d7_4fe5_8d63_3324458dc4ca.slice/crio-fbbe63324a25dc8143b5a90485162baa9c35f19503fcce6ed53dd38a061eadf6 WatchSource:0}: Error finding container fbbe63324a25dc8143b5a90485162baa9c35f19503fcce6ed53dd38a061eadf6: Status 404 returned error can't find the container with id fbbe63324a25dc8143b5a90485162baa9c35f19503fcce6ed53dd38a061eadf6 Jul 22 03:26:14.343006 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:14.342919 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" event={"ID":"d14af780-53d7-4fe5-8d63-3324458dc4ca","Type":"ContainerStarted","Data":"fbbe63324a25dc8143b5a90485162baa9c35f19503fcce6ed53dd38a061eadf6"} Jul 22 03:26:15.350307 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:15.350260 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" event={"ID":"d14af780-53d7-4fe5-8d63-3324458dc4ca","Type":"ContainerStarted","Data":"f194149e8fd9e8ba5064d3cf8e462818354f6116d92b470c8c0429460053f570"} Jul 22 03:26:15.350699 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:15.350439 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" Jul 22 03:26:15.370141 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:15.370092 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" podStartSLOduration=2.999321413 podStartE2EDuration="3.370077115s" totalImagesPullingTime="370.755702ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:26:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:26:15.368228101 +0000 UTC m=+1704.559422404" watchObservedRunningTime="2026-07-22 03:26:15.370077115 +0000 UTC m=+1704.561271417" Jul 22 03:26:21.360386 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:21.360358 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" Jul 22 03:26:49.418823 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:49.418789 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww"] Jul 22 03:26:49.419693 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:49.418982 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" podUID="d14af780-53d7-4fe5-8d63-3324458dc4ca" containerName="maas-api" containerID="cri-o://f194149e8fd9e8ba5064d3cf8e462818354f6116d92b470c8c0429460053f570" gracePeriod=30 Jul 22 03:26:49.662803 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:49.662776 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" Jul 22 03:26:49.834588 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:49.834517 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d14af780-53d7-4fe5-8d63-3324458dc4ca-maas-api-tls\") pod \"d14af780-53d7-4fe5-8d63-3324458dc4ca\" (UID: \"d14af780-53d7-4fe5-8d63-3324458dc4ca\") " Jul 22 03:26:49.834588 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:49.834572 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d14af780-53d7-4fe5-8d63-3324458dc4ca-openshift-service-ca\") pod \"d14af780-53d7-4fe5-8d63-3324458dc4ca\" (UID: \"d14af780-53d7-4fe5-8d63-3324458dc4ca\") " Jul 22 03:26:49.834756 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:49.834603 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jb56k\" (UniqueName: \"kubernetes.io/projected/d14af780-53d7-4fe5-8d63-3324458dc4ca-kube-api-access-jb56k\") pod \"d14af780-53d7-4fe5-8d63-3324458dc4ca\" (UID: \"d14af780-53d7-4fe5-8d63-3324458dc4ca\") " Jul 22 03:26:49.834934 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:49.834906 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d14af780-53d7-4fe5-8d63-3324458dc4ca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d14af780-53d7-4fe5-8d63-3324458dc4ca" (UID: "d14af780-53d7-4fe5-8d63-3324458dc4ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:26:49.836382 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:49.836351 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d14af780-53d7-4fe5-8d63-3324458dc4ca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d14af780-53d7-4fe5-8d63-3324458dc4ca" (UID: "d14af780-53d7-4fe5-8d63-3324458dc4ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:26:49.836583 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:49.836563 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d14af780-53d7-4fe5-8d63-3324458dc4ca-kube-api-access-jb56k" (OuterVolumeSpecName: "kube-api-access-jb56k") pod "d14af780-53d7-4fe5-8d63-3324458dc4ca" (UID: "d14af780-53d7-4fe5-8d63-3324458dc4ca"). InnerVolumeSpecName "kube-api-access-jb56k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:26:49.936171 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:49.936147 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d14af780-53d7-4fe5-8d63-3324458dc4ca-maas-api-tls\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:26:49.936171 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:49.936169 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d14af780-53d7-4fe5-8d63-3324458dc4ca-openshift-service-ca\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:26:49.936295 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:49.936180 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jb56k\" (UniqueName: \"kubernetes.io/projected/d14af780-53d7-4fe5-8d63-3324458dc4ca-kube-api-access-jb56k\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:26:50.511215 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:50.511177 2580 generic.go:356] "Generic (PLEG): container finished" podID="d14af780-53d7-4fe5-8d63-3324458dc4ca" containerID="f194149e8fd9e8ba5064d3cf8e462818354f6116d92b470c8c0429460053f570" exitCode=0 Jul 22 03:26:50.511660 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:50.511219 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" Jul 22 03:26:50.511660 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:50.511272 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" event={"ID":"d14af780-53d7-4fe5-8d63-3324458dc4ca","Type":"ContainerDied","Data":"f194149e8fd9e8ba5064d3cf8e462818354f6116d92b470c8c0429460053f570"} Jul 22 03:26:50.511660 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:50.511322 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww" event={"ID":"d14af780-53d7-4fe5-8d63-3324458dc4ca","Type":"ContainerDied","Data":"fbbe63324a25dc8143b5a90485162baa9c35f19503fcce6ed53dd38a061eadf6"} Jul 22 03:26:50.511660 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:50.511346 2580 scope.go:117] "RemoveContainer" containerID="f194149e8fd9e8ba5064d3cf8e462818354f6116d92b470c8c0429460053f570" Jul 22 03:26:50.519982 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:50.519964 2580 scope.go:117] "RemoveContainer" containerID="f194149e8fd9e8ba5064d3cf8e462818354f6116d92b470c8c0429460053f570" Jul 22 03:26:50.520232 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:26:50.520210 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f194149e8fd9e8ba5064d3cf8e462818354f6116d92b470c8c0429460053f570\": container with ID starting with f194149e8fd9e8ba5064d3cf8e462818354f6116d92b470c8c0429460053f570 not found: ID does not exist" containerID="f194149e8fd9e8ba5064d3cf8e462818354f6116d92b470c8c0429460053f570" Jul 22 03:26:50.520279 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:50.520245 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f194149e8fd9e8ba5064d3cf8e462818354f6116d92b470c8c0429460053f570"} err="failed to get container status \"f194149e8fd9e8ba5064d3cf8e462818354f6116d92b470c8c0429460053f570\": rpc error: code = NotFound desc = could not find container \"f194149e8fd9e8ba5064d3cf8e462818354f6116d92b470c8c0429460053f570\": container with ID starting with f194149e8fd9e8ba5064d3cf8e462818354f6116d92b470c8c0429460053f570 not found: ID does not exist" Jul 22 03:26:50.534419 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:50.534393 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww"] Jul 22 03:26:50.537676 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:50.537657 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-67a1aa21-559dd9cb57-p6tww"] Jul 22 03:26:51.367580 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:51.367547 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d14af780-53d7-4fe5-8d63-3324458dc4ca" path="/var/lib/kubelet/pods/d14af780-53d7-4fe5-8d63-3324458dc4ca/volumes" Jul 22 03:26:58.200229 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.200196 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d"] Jul 22 03:26:58.200843 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.200623 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d14af780-53d7-4fe5-8d63-3324458dc4ca" containerName="maas-api" Jul 22 03:26:58.200843 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.200638 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d14af780-53d7-4fe5-8d63-3324458dc4ca" containerName="maas-api" Jul 22 03:26:58.200843 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.200721 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d14af780-53d7-4fe5-8d63-3324458dc4ca" containerName="maas-api" Jul 22 03:26:58.203741 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.203725 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" Jul 22 03:26:58.206184 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.206165 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-7df930d3-serving-cert\"" Jul 22 03:26:58.212616 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.212594 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d"] Jul 22 03:26:58.309388 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.309185 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ce52fb4-d945-43ce-802a-91be991d2e2d-openshift-service-ca\") pod \"maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d\" (UID: \"8ce52fb4-d945-43ce-802a-91be991d2e2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" Jul 22 03:26:58.309388 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.309237 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d8bm\" (UniqueName: \"kubernetes.io/projected/8ce52fb4-d945-43ce-802a-91be991d2e2d-kube-api-access-9d8bm\") pod \"maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d\" (UID: \"8ce52fb4-d945-43ce-802a-91be991d2e2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" Jul 22 03:26:58.309388 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.309269 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ce52fb4-d945-43ce-802a-91be991d2e2d-maas-api-tls\") pod \"maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d\" (UID: \"8ce52fb4-d945-43ce-802a-91be991d2e2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" Jul 22 03:26:58.410049 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.410015 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ce52fb4-d945-43ce-802a-91be991d2e2d-openshift-service-ca\") pod \"maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d\" (UID: \"8ce52fb4-d945-43ce-802a-91be991d2e2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" Jul 22 03:26:58.410225 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.410059 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d8bm\" (UniqueName: \"kubernetes.io/projected/8ce52fb4-d945-43ce-802a-91be991d2e2d-kube-api-access-9d8bm\") pod \"maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d\" (UID: \"8ce52fb4-d945-43ce-802a-91be991d2e2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" Jul 22 03:26:58.410225 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.410099 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ce52fb4-d945-43ce-802a-91be991d2e2d-maas-api-tls\") pod \"maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d\" (UID: \"8ce52fb4-d945-43ce-802a-91be991d2e2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" Jul 22 03:26:58.410679 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.410647 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ce52fb4-d945-43ce-802a-91be991d2e2d-openshift-service-ca\") pod \"maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d\" (UID: \"8ce52fb4-d945-43ce-802a-91be991d2e2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" Jul 22 03:26:58.412516 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.412494 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ce52fb4-d945-43ce-802a-91be991d2e2d-maas-api-tls\") pod \"maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d\" (UID: \"8ce52fb4-d945-43ce-802a-91be991d2e2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" Jul 22 03:26:58.419005 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.418983 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d8bm\" (UniqueName: \"kubernetes.io/projected/8ce52fb4-d945-43ce-802a-91be991d2e2d-kube-api-access-9d8bm\") pod \"maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d\" (UID: \"8ce52fb4-d945-43ce-802a-91be991d2e2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" Jul 22 03:26:58.515188 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.515117 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" Jul 22 03:26:58.648467 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:58.648443 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d"] Jul 22 03:26:58.650840 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:26:58.650800 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ce52fb4_d945_43ce_802a_91be991d2e2d.slice/crio-c32cbd414abe66fc2448d1b723683d13cfc7cfc2b29238289c460a42c489028b WatchSource:0}: Error finding container c32cbd414abe66fc2448d1b723683d13cfc7cfc2b29238289c460a42c489028b: Status 404 returned error can't find the container with id c32cbd414abe66fc2448d1b723683d13cfc7cfc2b29238289c460a42c489028b Jul 22 03:26:59.555534 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:59.555457 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" event={"ID":"8ce52fb4-d945-43ce-802a-91be991d2e2d","Type":"ContainerStarted","Data":"743e7b0179f3c4c571ccfe48021ad5e199c7e3b7138a527cb38334b4b54b20e8"} Jul 22 03:26:59.555534 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:59.555522 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" Jul 22 03:26:59.555927 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:59.555540 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" event={"ID":"8ce52fb4-d945-43ce-802a-91be991d2e2d","Type":"ContainerStarted","Data":"c32cbd414abe66fc2448d1b723683d13cfc7cfc2b29238289c460a42c489028b"} Jul 22 03:26:59.576068 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:26:59.576020 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" podStartSLOduration=1.116902458 podStartE2EDuration="1.576002045s" totalImagesPullingTime="459.099587ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:26:59.575033679 +0000 UTC m=+1748.766228029" watchObservedRunningTime="2026-07-22 03:26:59.576002045 +0000 UTC m=+1748.767196352" Jul 22 03:27:05.566561 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:05.566528 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" Jul 22 03:27:33.776823 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:33.776790 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d"] Jul 22 03:27:33.777348 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:33.777014 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" podUID="8ce52fb4-d945-43ce-802a-91be991d2e2d" containerName="maas-api" containerID="cri-o://743e7b0179f3c4c571ccfe48021ad5e199c7e3b7138a527cb38334b4b54b20e8" gracePeriod=30 Jul 22 03:27:34.025419 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.025395 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" Jul 22 03:27:34.116058 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.116031 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ce52fb4-d945-43ce-802a-91be991d2e2d-maas-api-tls\") pod \"8ce52fb4-d945-43ce-802a-91be991d2e2d\" (UID: \"8ce52fb4-d945-43ce-802a-91be991d2e2d\") " Jul 22 03:27:34.116227 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.116116 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9d8bm\" (UniqueName: \"kubernetes.io/projected/8ce52fb4-d945-43ce-802a-91be991d2e2d-kube-api-access-9d8bm\") pod \"8ce52fb4-d945-43ce-802a-91be991d2e2d\" (UID: \"8ce52fb4-d945-43ce-802a-91be991d2e2d\") " Jul 22 03:27:34.116227 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.116199 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ce52fb4-d945-43ce-802a-91be991d2e2d-openshift-service-ca\") pod \"8ce52fb4-d945-43ce-802a-91be991d2e2d\" (UID: \"8ce52fb4-d945-43ce-802a-91be991d2e2d\") " Jul 22 03:27:34.116571 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.116541 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ce52fb4-d945-43ce-802a-91be991d2e2d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8ce52fb4-d945-43ce-802a-91be991d2e2d" (UID: "8ce52fb4-d945-43ce-802a-91be991d2e2d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:27:34.118213 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.118186 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ce52fb4-d945-43ce-802a-91be991d2e2d-kube-api-access-9d8bm" (OuterVolumeSpecName: "kube-api-access-9d8bm") pod "8ce52fb4-d945-43ce-802a-91be991d2e2d" (UID: "8ce52fb4-d945-43ce-802a-91be991d2e2d"). InnerVolumeSpecName "kube-api-access-9d8bm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:27:34.118328 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.118251 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ce52fb4-d945-43ce-802a-91be991d2e2d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8ce52fb4-d945-43ce-802a-91be991d2e2d" (UID: "8ce52fb4-d945-43ce-802a-91be991d2e2d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:27:34.217787 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.217759 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ce52fb4-d945-43ce-802a-91be991d2e2d-openshift-service-ca\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:27:34.217787 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.217784 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8ce52fb4-d945-43ce-802a-91be991d2e2d-maas-api-tls\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:27:34.217957 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.217797 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d8bm\" (UniqueName: \"kubernetes.io/projected/8ce52fb4-d945-43ce-802a-91be991d2e2d-kube-api-access-9d8bm\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:27:34.723146 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.723109 2580 generic.go:356] "Generic (PLEG): container finished" podID="8ce52fb4-d945-43ce-802a-91be991d2e2d" containerID="743e7b0179f3c4c571ccfe48021ad5e199c7e3b7138a527cb38334b4b54b20e8" exitCode=0 Jul 22 03:27:34.723146 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.723141 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" event={"ID":"8ce52fb4-d945-43ce-802a-91be991d2e2d","Type":"ContainerDied","Data":"743e7b0179f3c4c571ccfe48021ad5e199c7e3b7138a527cb38334b4b54b20e8"} Jul 22 03:27:34.723395 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.723194 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" event={"ID":"8ce52fb4-d945-43ce-802a-91be991d2e2d","Type":"ContainerDied","Data":"c32cbd414abe66fc2448d1b723683d13cfc7cfc2b29238289c460a42c489028b"} Jul 22 03:27:34.723395 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.723217 2580 scope.go:117] "RemoveContainer" containerID="743e7b0179f3c4c571ccfe48021ad5e199c7e3b7138a527cb38334b4b54b20e8" Jul 22 03:27:34.723395 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.723153 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d" Jul 22 03:27:34.732558 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.732539 2580 scope.go:117] "RemoveContainer" containerID="743e7b0179f3c4c571ccfe48021ad5e199c7e3b7138a527cb38334b4b54b20e8" Jul 22 03:27:34.732801 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:27:34.732780 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"743e7b0179f3c4c571ccfe48021ad5e199c7e3b7138a527cb38334b4b54b20e8\": container with ID starting with 743e7b0179f3c4c571ccfe48021ad5e199c7e3b7138a527cb38334b4b54b20e8 not found: ID does not exist" containerID="743e7b0179f3c4c571ccfe48021ad5e199c7e3b7138a527cb38334b4b54b20e8" Jul 22 03:27:34.732918 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.732809 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"743e7b0179f3c4c571ccfe48021ad5e199c7e3b7138a527cb38334b4b54b20e8"} err="failed to get container status \"743e7b0179f3c4c571ccfe48021ad5e199c7e3b7138a527cb38334b4b54b20e8\": rpc error: code = NotFound desc = could not find container \"743e7b0179f3c4c571ccfe48021ad5e199c7e3b7138a527cb38334b4b54b20e8\": container with ID starting with 743e7b0179f3c4c571ccfe48021ad5e199c7e3b7138a527cb38334b4b54b20e8 not found: ID does not exist" Jul 22 03:27:34.752541 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.752518 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d"] Jul 22 03:27:34.762269 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:34.762246 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7df930d3-6ccf7c49cb-c587d"] Jul 22 03:27:35.366899 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:35.366871 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ce52fb4-d945-43ce-802a-91be991d2e2d" path="/var/lib/kubelet/pods/8ce52fb4-d945-43ce-802a-91be991d2e2d/volumes" Jul 22 03:27:51.373641 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:51.373609 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:27:51.378046 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:51.378024 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:27:51.379700 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:51.379681 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:27:51.383993 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:27:51.383975 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:29:26.625160 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:26.625074 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7"] Jul 22 03:29:26.625769 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:26.625411 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ce52fb4-d945-43ce-802a-91be991d2e2d" containerName="maas-api" Jul 22 03:29:26.625769 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:26.625422 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ce52fb4-d945-43ce-802a-91be991d2e2d" containerName="maas-api" Jul 22 03:29:26.625769 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:26.625489 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ce52fb4-d945-43ce-802a-91be991d2e2d" containerName="maas-api" Jul 22 03:29:26.628469 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:26.628450 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" Jul 22 03:29:26.631081 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:26.631052 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-fe15ed44-serving-cert\"" Jul 22 03:29:26.640872 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:26.640846 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7"] Jul 22 03:29:26.765466 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:26.765439 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lt9r\" (UniqueName: \"kubernetes.io/projected/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-kube-api-access-4lt9r\") pod \"maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7\" (UID: \"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" Jul 22 03:29:26.765610 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:26.765497 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-maas-api-tls\") pod \"maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7\" (UID: \"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" Jul 22 03:29:26.765610 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:26.765552 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-openshift-service-ca\") pod \"maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7\" (UID: \"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" Jul 22 03:29:26.866186 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:26.866160 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-maas-api-tls\") pod \"maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7\" (UID: \"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" Jul 22 03:29:26.866335 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:26.866214 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-openshift-service-ca\") pod \"maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7\" (UID: \"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" Jul 22 03:29:26.866335 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:26.866256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lt9r\" (UniqueName: \"kubernetes.io/projected/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-kube-api-access-4lt9r\") pod \"maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7\" (UID: \"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" Jul 22 03:29:26.866335 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:29:26.866306 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-serving-cert: secret "maas-api-e2e-mt-fe15ed44-serving-cert" not found Jul 22 03:29:26.866462 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:29:26.866368 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-maas-api-tls podName:1d1c42e2-38eb-4e16-9e79-c0c9db263ec0 nodeName:}" failed. No retries permitted until 2026-07-22 03:29:27.366350349 +0000 UTC m=+1896.557544630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-maas-api-tls") pod "maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" (UID: "1d1c42e2-38eb-4e16-9e79-c0c9db263ec0") : secret "maas-api-e2e-mt-fe15ed44-serving-cert" not found Jul 22 03:29:26.866886 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:26.866869 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-openshift-service-ca\") pod \"maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7\" (UID: \"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" Jul 22 03:29:26.875050 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:26.875025 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lt9r\" (UniqueName: \"kubernetes.io/projected/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-kube-api-access-4lt9r\") pod \"maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7\" (UID: \"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" Jul 22 03:29:27.370745 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:27.370712 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-maas-api-tls\") pod \"maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7\" (UID: \"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" Jul 22 03:29:27.373130 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:27.373107 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-maas-api-tls\") pod \"maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7\" (UID: \"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" Jul 22 03:29:27.541711 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:27.541683 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" Jul 22 03:29:27.670922 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:27.670897 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7"] Jul 22 03:29:27.673217 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:29:27.673187 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d1c42e2_38eb_4e16_9e79_c0c9db263ec0.slice/crio-8d1f4adf276c047fcd0dbaafa38df76f7f379397655b8493c990449f47ab132d WatchSource:0}: Error finding container 8d1f4adf276c047fcd0dbaafa38df76f7f379397655b8493c990449f47ab132d: Status 404 returned error can't find the container with id 8d1f4adf276c047fcd0dbaafa38df76f7f379397655b8493c990449f47ab132d Jul 22 03:29:28.241586 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:28.241555 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" event={"ID":"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0","Type":"ContainerStarted","Data":"8d1f4adf276c047fcd0dbaafa38df76f7f379397655b8493c990449f47ab132d"} Jul 22 03:29:29.248073 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:29.248038 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" event={"ID":"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0","Type":"ContainerStarted","Data":"8f442185d8e68bdabd46c873556d50895ac143212cdb5193c9451c5063d6f4c0"} Jul 22 03:29:29.248447 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:29.248105 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" Jul 22 03:29:29.267776 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:29.267731 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" podStartSLOduration=2.741105775 podStartE2EDuration="3.26771631s" totalImagesPullingTime="526.610535ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:29:29.266924001 +0000 UTC m=+1898.458118343" watchObservedRunningTime="2026-07-22 03:29:29.26771631 +0000 UTC m=+1898.458910611" Jul 22 03:29:35.259293 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:35.259265 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" Jul 22 03:29:49.494482 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:49.494451 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7"] Jul 22 03:29:49.494997 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:49.494655 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" podUID="1d1c42e2-38eb-4e16-9e79-c0c9db263ec0" containerName="maas-api" containerID="cri-o://8f442185d8e68bdabd46c873556d50895ac143212cdb5193c9451c5063d6f4c0" gracePeriod=30 Jul 22 03:29:49.748823 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:49.748761 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" Jul 22 03:29:49.861254 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:49.861219 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-maas-api-tls\") pod \"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0\" (UID: \"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0\") " Jul 22 03:29:49.861446 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:49.861320 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-openshift-service-ca\") pod \"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0\" (UID: \"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0\") " Jul 22 03:29:49.861446 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:49.861372 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lt9r\" (UniqueName: \"kubernetes.io/projected/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-kube-api-access-4lt9r\") pod \"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0\" (UID: \"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0\") " Jul 22 03:29:49.861709 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:49.861683 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1d1c42e2-38eb-4e16-9e79-c0c9db263ec0" (UID: "1d1c42e2-38eb-4e16-9e79-c0c9db263ec0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:29:49.863257 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:49.863231 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1d1c42e2-38eb-4e16-9e79-c0c9db263ec0" (UID: "1d1c42e2-38eb-4e16-9e79-c0c9db263ec0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:29:49.863364 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:49.863345 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-kube-api-access-4lt9r" (OuterVolumeSpecName: "kube-api-access-4lt9r") pod "1d1c42e2-38eb-4e16-9e79-c0c9db263ec0" (UID: "1d1c42e2-38eb-4e16-9e79-c0c9db263ec0"). InnerVolumeSpecName "kube-api-access-4lt9r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:29:49.962537 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:49.962508 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-maas-api-tls\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:29:49.962537 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:49.962536 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-openshift-service-ca\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:29:49.962709 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:49.962546 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lt9r\" (UniqueName: \"kubernetes.io/projected/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0-kube-api-access-4lt9r\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:29:50.340928 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:50.340894 2580 generic.go:356] "Generic (PLEG): container finished" podID="1d1c42e2-38eb-4e16-9e79-c0c9db263ec0" containerID="8f442185d8e68bdabd46c873556d50895ac143212cdb5193c9451c5063d6f4c0" exitCode=0 Jul 22 03:29:50.341088 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:50.340935 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" Jul 22 03:29:50.341088 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:50.340976 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" event={"ID":"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0","Type":"ContainerDied","Data":"8f442185d8e68bdabd46c873556d50895ac143212cdb5193c9451c5063d6f4c0"} Jul 22 03:29:50.341088 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:50.341029 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7" event={"ID":"1d1c42e2-38eb-4e16-9e79-c0c9db263ec0","Type":"ContainerDied","Data":"8d1f4adf276c047fcd0dbaafa38df76f7f379397655b8493c990449f47ab132d"} Jul 22 03:29:50.341088 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:50.341058 2580 scope.go:117] "RemoveContainer" containerID="8f442185d8e68bdabd46c873556d50895ac143212cdb5193c9451c5063d6f4c0" Jul 22 03:29:50.349358 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:50.349342 2580 scope.go:117] "RemoveContainer" containerID="8f442185d8e68bdabd46c873556d50895ac143212cdb5193c9451c5063d6f4c0" Jul 22 03:29:50.349604 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:29:50.349585 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f442185d8e68bdabd46c873556d50895ac143212cdb5193c9451c5063d6f4c0\": container with ID starting with 8f442185d8e68bdabd46c873556d50895ac143212cdb5193c9451c5063d6f4c0 not found: ID does not exist" containerID="8f442185d8e68bdabd46c873556d50895ac143212cdb5193c9451c5063d6f4c0" Jul 22 03:29:50.349647 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:50.349613 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f442185d8e68bdabd46c873556d50895ac143212cdb5193c9451c5063d6f4c0"} err="failed to get container status \"8f442185d8e68bdabd46c873556d50895ac143212cdb5193c9451c5063d6f4c0\": rpc error: code = NotFound desc = could not find container \"8f442185d8e68bdabd46c873556d50895ac143212cdb5193c9451c5063d6f4c0\": container with ID starting with 8f442185d8e68bdabd46c873556d50895ac143212cdb5193c9451c5063d6f4c0 not found: ID does not exist" Jul 22 03:29:50.361913 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:50.361884 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7"] Jul 22 03:29:50.366561 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:50.366541 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fe15ed44-8b5b6d4c6-kmll7"] Jul 22 03:29:51.366433 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:29:51.366399 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d1c42e2-38eb-4e16-9e79-c0c9db263ec0" path="/var/lib/kubelet/pods/1d1c42e2-38eb-4e16-9e79-c0c9db263ec0/volumes" Jul 22 03:30:24.646658 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:24.646623 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls"] Jul 22 03:30:24.647127 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:24.646999 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d1c42e2-38eb-4e16-9e79-c0c9db263ec0" containerName="maas-api" Jul 22 03:30:24.647127 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:24.647011 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d1c42e2-38eb-4e16-9e79-c0c9db263ec0" containerName="maas-api" Jul 22 03:30:24.647127 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:24.647068 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d1c42e2-38eb-4e16-9e79-c0c9db263ec0" containerName="maas-api" Jul 22 03:30:24.650161 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:24.650140 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" Jul 22 03:30:24.652285 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:24.652265 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-017b0f74-serving-cert\"" Jul 22 03:30:24.659713 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:24.659694 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls"] Jul 22 03:30:24.765296 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:24.765260 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppg67\" (UniqueName: \"kubernetes.io/projected/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-kube-api-access-ppg67\") pod \"maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls\" (UID: \"67dc04bd-bb6c-443e-8a00-a9557a84b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" Jul 22 03:30:24.765457 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:24.765381 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-maas-api-tls\") pod \"maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls\" (UID: \"67dc04bd-bb6c-443e-8a00-a9557a84b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" Jul 22 03:30:24.765457 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:24.765448 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-openshift-service-ca\") pod \"maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls\" (UID: \"67dc04bd-bb6c-443e-8a00-a9557a84b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" Jul 22 03:30:24.866310 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:24.866273 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-maas-api-tls\") pod \"maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls\" (UID: \"67dc04bd-bb6c-443e-8a00-a9557a84b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" Jul 22 03:30:24.866481 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:24.866336 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-openshift-service-ca\") pod \"maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls\" (UID: \"67dc04bd-bb6c-443e-8a00-a9557a84b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" Jul 22 03:30:24.866481 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:30:24.866414 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-serving-cert: secret "maas-api-e2e-mt-017b0f74-serving-cert" not found Jul 22 03:30:24.866481 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:30:24.866475 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-maas-api-tls podName:67dc04bd-bb6c-443e-8a00-a9557a84b3b5 nodeName:}" failed. No retries permitted until 2026-07-22 03:30:25.366455935 +0000 UTC m=+1954.557650216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-maas-api-tls") pod "maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" (UID: "67dc04bd-bb6c-443e-8a00-a9557a84b3b5") : secret "maas-api-e2e-mt-017b0f74-serving-cert" not found Jul 22 03:30:24.866481 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:24.866422 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppg67\" (UniqueName: \"kubernetes.io/projected/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-kube-api-access-ppg67\") pod \"maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls\" (UID: \"67dc04bd-bb6c-443e-8a00-a9557a84b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" Jul 22 03:30:24.867037 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:24.867016 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-openshift-service-ca\") pod \"maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls\" (UID: \"67dc04bd-bb6c-443e-8a00-a9557a84b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" Jul 22 03:30:24.876433 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:24.876411 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppg67\" (UniqueName: \"kubernetes.io/projected/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-kube-api-access-ppg67\") pod \"maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls\" (UID: \"67dc04bd-bb6c-443e-8a00-a9557a84b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" Jul 22 03:30:25.371367 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:25.371335 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-maas-api-tls\") pod \"maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls\" (UID: \"67dc04bd-bb6c-443e-8a00-a9557a84b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" Jul 22 03:30:25.373618 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:25.373599 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-maas-api-tls\") pod \"maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls\" (UID: \"67dc04bd-bb6c-443e-8a00-a9557a84b3b5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" Jul 22 03:30:25.560792 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:25.560759 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" Jul 22 03:30:25.691624 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:25.691599 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls"] Jul 22 03:30:25.693802 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:30:25.693779 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67dc04bd_bb6c_443e_8a00_a9557a84b3b5.slice/crio-6663c8e68b37d23ceb6fd0b882f468d209ce2cb63b99d453baff26aa9797d4dd WatchSource:0}: Error finding container 6663c8e68b37d23ceb6fd0b882f468d209ce2cb63b99d453baff26aa9797d4dd: Status 404 returned error can't find the container with id 6663c8e68b37d23ceb6fd0b882f468d209ce2cb63b99d453baff26aa9797d4dd Jul 22 03:30:25.695487 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:25.695467 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 03:30:26.500613 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:26.500571 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" event={"ID":"67dc04bd-bb6c-443e-8a00-a9557a84b3b5","Type":"ContainerStarted","Data":"b56410bca00d76f5c873da0198e17d6a5215ea0ac5c976a46ee68e65c1ebc272"} Jul 22 03:30:26.500807 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:26.500660 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" event={"ID":"67dc04bd-bb6c-443e-8a00-a9557a84b3b5","Type":"ContainerStarted","Data":"6663c8e68b37d23ceb6fd0b882f468d209ce2cb63b99d453baff26aa9797d4dd"} Jul 22 03:30:26.500807 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:26.500714 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" Jul 22 03:30:26.533283 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:26.533233 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" podStartSLOduration=2.193278063 podStartE2EDuration="2.533219731s" totalImagesPullingTime="339.941668ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:30:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:30:26.531097993 +0000 UTC m=+1955.722292313" watchObservedRunningTime="2026-07-22 03:30:26.533219731 +0000 UTC m=+1955.724414033" Jul 22 03:30:32.511529 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:30:32.511497 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" Jul 22 03:31:23.289741 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.289709 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls"] Jul 22 03:31:23.290281 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.290002 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" podUID="67dc04bd-bb6c-443e-8a00-a9557a84b3b5" containerName="maas-api" containerID="cri-o://b56410bca00d76f5c873da0198e17d6a5215ea0ac5c976a46ee68e65c1ebc272" gracePeriod=30 Jul 22 03:31:23.547116 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.547052 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" Jul 22 03:31:23.583226 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.583193 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppg67\" (UniqueName: \"kubernetes.io/projected/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-kube-api-access-ppg67\") pod \"67dc04bd-bb6c-443e-8a00-a9557a84b3b5\" (UID: \"67dc04bd-bb6c-443e-8a00-a9557a84b3b5\") " Jul 22 03:31:23.583226 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.583226 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-openshift-service-ca\") pod \"67dc04bd-bb6c-443e-8a00-a9557a84b3b5\" (UID: \"67dc04bd-bb6c-443e-8a00-a9557a84b3b5\") " Jul 22 03:31:23.583449 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.583307 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-maas-api-tls\") pod \"67dc04bd-bb6c-443e-8a00-a9557a84b3b5\" (UID: \"67dc04bd-bb6c-443e-8a00-a9557a84b3b5\") " Jul 22 03:31:23.583691 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.583666 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "67dc04bd-bb6c-443e-8a00-a9557a84b3b5" (UID: "67dc04bd-bb6c-443e-8a00-a9557a84b3b5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:31:23.585543 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.585513 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "67dc04bd-bb6c-443e-8a00-a9557a84b3b5" (UID: "67dc04bd-bb6c-443e-8a00-a9557a84b3b5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:31:23.585664 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.585558 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-kube-api-access-ppg67" (OuterVolumeSpecName: "kube-api-access-ppg67") pod "67dc04bd-bb6c-443e-8a00-a9557a84b3b5" (UID: "67dc04bd-bb6c-443e-8a00-a9557a84b3b5"). InnerVolumeSpecName "kube-api-access-ppg67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:31:23.684464 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.684434 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-maas-api-tls\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:31:23.684464 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.684461 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppg67\" (UniqueName: \"kubernetes.io/projected/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-kube-api-access-ppg67\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:31:23.684464 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.684475 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/67dc04bd-bb6c-443e-8a00-a9557a84b3b5-openshift-service-ca\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:31:23.760916 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.760879 2580 generic.go:356] "Generic (PLEG): container finished" podID="67dc04bd-bb6c-443e-8a00-a9557a84b3b5" containerID="b56410bca00d76f5c873da0198e17d6a5215ea0ac5c976a46ee68e65c1ebc272" exitCode=0 Jul 22 03:31:23.761082 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.760924 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" Jul 22 03:31:23.761082 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.760968 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" event={"ID":"67dc04bd-bb6c-443e-8a00-a9557a84b3b5","Type":"ContainerDied","Data":"b56410bca00d76f5c873da0198e17d6a5215ea0ac5c976a46ee68e65c1ebc272"} Jul 22 03:31:23.761082 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.761023 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls" event={"ID":"67dc04bd-bb6c-443e-8a00-a9557a84b3b5","Type":"ContainerDied","Data":"6663c8e68b37d23ceb6fd0b882f468d209ce2cb63b99d453baff26aa9797d4dd"} Jul 22 03:31:23.761082 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.761045 2580 scope.go:117] "RemoveContainer" containerID="b56410bca00d76f5c873da0198e17d6a5215ea0ac5c976a46ee68e65c1ebc272" Jul 22 03:31:23.770299 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.770277 2580 scope.go:117] "RemoveContainer" containerID="b56410bca00d76f5c873da0198e17d6a5215ea0ac5c976a46ee68e65c1ebc272" Jul 22 03:31:23.770551 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:31:23.770531 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b56410bca00d76f5c873da0198e17d6a5215ea0ac5c976a46ee68e65c1ebc272\": container with ID starting with b56410bca00d76f5c873da0198e17d6a5215ea0ac5c976a46ee68e65c1ebc272 not found: ID does not exist" containerID="b56410bca00d76f5c873da0198e17d6a5215ea0ac5c976a46ee68e65c1ebc272" Jul 22 03:31:23.770617 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.770563 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b56410bca00d76f5c873da0198e17d6a5215ea0ac5c976a46ee68e65c1ebc272"} err="failed to get container status \"b56410bca00d76f5c873da0198e17d6a5215ea0ac5c976a46ee68e65c1ebc272\": rpc error: code = NotFound desc = could not find container \"b56410bca00d76f5c873da0198e17d6a5215ea0ac5c976a46ee68e65c1ebc272\": container with ID starting with b56410bca00d76f5c873da0198e17d6a5215ea0ac5c976a46ee68e65c1ebc272 not found: ID does not exist" Jul 22 03:31:23.784561 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.784531 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls"] Jul 22 03:31:23.788640 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:23.788617 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-017b0f74-6d6dddb9bf-r2gls"] Jul 22 03:31:25.367401 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:31:25.367357 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67dc04bd-bb6c-443e-8a00-a9557a84b3b5" path="/var/lib/kubelet/pods/67dc04bd-bb6c-443e-8a00-a9557a84b3b5/volumes" Jul 22 03:32:51.401642 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:32:51.401607 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:32:51.407211 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:32:51.407190 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:32:51.408197 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:32:51.408175 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:32:51.413590 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:32:51.413571 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:33:18.495526 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.495477 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm"] Jul 22 03:33:18.496461 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.496438 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67dc04bd-bb6c-443e-8a00-a9557a84b3b5" containerName="maas-api" Jul 22 03:33:18.496533 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.496465 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="67dc04bd-bb6c-443e-8a00-a9557a84b3b5" containerName="maas-api" Jul 22 03:33:18.496669 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.496656 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="67dc04bd-bb6c-443e-8a00-a9557a84b3b5" containerName="maas-api" Jul 22 03:33:18.500586 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.500557 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" Jul 22 03:33:18.503735 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.503705 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-fd4878-serving-cert\"" Jul 22 03:33:18.510621 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.510594 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm"] Jul 22 03:33:18.628965 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.628932 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e282274b-0b60-4451-8ce2-a9ff9c6754c2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm\" (UID: \"e282274b-0b60-4451-8ce2-a9ff9c6754c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" Jul 22 03:33:18.629134 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.628973 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkm5n\" (UniqueName: \"kubernetes.io/projected/e282274b-0b60-4451-8ce2-a9ff9c6754c2-kube-api-access-lkm5n\") pod \"maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm\" (UID: \"e282274b-0b60-4451-8ce2-a9ff9c6754c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" Jul 22 03:33:18.629134 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.629086 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e282274b-0b60-4451-8ce2-a9ff9c6754c2-maas-api-tls\") pod \"maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm\" (UID: \"e282274b-0b60-4451-8ce2-a9ff9c6754c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" Jul 22 03:33:18.730485 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.730459 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e282274b-0b60-4451-8ce2-a9ff9c6754c2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm\" (UID: \"e282274b-0b60-4451-8ce2-a9ff9c6754c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" Jul 22 03:33:18.730651 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.730492 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkm5n\" (UniqueName: \"kubernetes.io/projected/e282274b-0b60-4451-8ce2-a9ff9c6754c2-kube-api-access-lkm5n\") pod \"maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm\" (UID: \"e282274b-0b60-4451-8ce2-a9ff9c6754c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" Jul 22 03:33:18.730651 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.730598 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e282274b-0b60-4451-8ce2-a9ff9c6754c2-maas-api-tls\") pod \"maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm\" (UID: \"e282274b-0b60-4451-8ce2-a9ff9c6754c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" Jul 22 03:33:18.731203 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.731177 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e282274b-0b60-4451-8ce2-a9ff9c6754c2-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm\" (UID: \"e282274b-0b60-4451-8ce2-a9ff9c6754c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" Jul 22 03:33:18.732995 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.732978 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e282274b-0b60-4451-8ce2-a9ff9c6754c2-maas-api-tls\") pod \"maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm\" (UID: \"e282274b-0b60-4451-8ce2-a9ff9c6754c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" Jul 22 03:33:18.738585 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.738565 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkm5n\" (UniqueName: \"kubernetes.io/projected/e282274b-0b60-4451-8ce2-a9ff9c6754c2-kube-api-access-lkm5n\") pod \"maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm\" (UID: \"e282274b-0b60-4451-8ce2-a9ff9c6754c2\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" Jul 22 03:33:18.815998 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.815931 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" Jul 22 03:33:18.949069 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:18.948982 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm"] Jul 22 03:33:18.954098 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:33:18.954059 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode282274b_0b60_4451_8ce2_a9ff9c6754c2.slice/crio-d9b6e073cf3bff55541e056a3979c6ede31b43096fba16530969fdd038710147 WatchSource:0}: Error finding container d9b6e073cf3bff55541e056a3979c6ede31b43096fba16530969fdd038710147: Status 404 returned error can't find the container with id d9b6e073cf3bff55541e056a3979c6ede31b43096fba16530969fdd038710147 Jul 22 03:33:19.297842 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:19.297795 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" event={"ID":"e282274b-0b60-4451-8ce2-a9ff9c6754c2","Type":"ContainerStarted","Data":"d9b6e073cf3bff55541e056a3979c6ede31b43096fba16530969fdd038710147"} Jul 22 03:33:20.303924 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:20.303892 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" event={"ID":"e282274b-0b60-4451-8ce2-a9ff9c6754c2","Type":"ContainerStarted","Data":"56a971c2ef596a04d0d215ade3825bc107d477f8fd227635ad0ff04b24976704"} Jul 22 03:33:20.304307 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:20.303979 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" Jul 22 03:33:20.321176 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:20.320953 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" podStartSLOduration=1.897189848 podStartE2EDuration="2.320936999s" totalImagesPullingTime="423.747151ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:20.320307821 +0000 UTC m=+2129.511502123" watchObservedRunningTime="2026-07-22 03:33:20.320936999 +0000 UTC m=+2129.512131304" Jul 22 03:33:26.313680 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:26.313652 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" Jul 22 03:33:26.618051 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:26.618017 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7"] Jul 22 03:33:26.621348 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:26.621328 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" Jul 22 03:33:26.624020 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:26.623998 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-30704e-serving-cert\"" Jul 22 03:33:26.633761 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:26.633739 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7"] Jul 22 03:33:26.707303 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:26.707276 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fz6g\" (UniqueName: \"kubernetes.io/projected/e79217db-5570-45db-b4bc-70d9cb68e0c9-kube-api-access-2fz6g\") pod \"maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7\" (UID: \"e79217db-5570-45db-b4bc-70d9cb68e0c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" Jul 22 03:33:26.707453 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:26.707325 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e79217db-5570-45db-b4bc-70d9cb68e0c9-maas-api-tls\") pod \"maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7\" (UID: \"e79217db-5570-45db-b4bc-70d9cb68e0c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" Jul 22 03:33:26.707453 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:26.707344 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e79217db-5570-45db-b4bc-70d9cb68e0c9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7\" (UID: \"e79217db-5570-45db-b4bc-70d9cb68e0c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" Jul 22 03:33:26.808097 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:26.808070 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fz6g\" (UniqueName: \"kubernetes.io/projected/e79217db-5570-45db-b4bc-70d9cb68e0c9-kube-api-access-2fz6g\") pod \"maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7\" (UID: \"e79217db-5570-45db-b4bc-70d9cb68e0c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" Jul 22 03:33:26.808261 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:26.808120 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e79217db-5570-45db-b4bc-70d9cb68e0c9-maas-api-tls\") pod \"maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7\" (UID: \"e79217db-5570-45db-b4bc-70d9cb68e0c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" Jul 22 03:33:26.808261 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:26.808145 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e79217db-5570-45db-b4bc-70d9cb68e0c9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7\" (UID: \"e79217db-5570-45db-b4bc-70d9cb68e0c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" Jul 22 03:33:26.808261 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:33:26.808238 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-serving-cert: secret "maas-api-e2e-inf-b-30704e-serving-cert" not found Jul 22 03:33:26.808431 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:33:26.808305 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e79217db-5570-45db-b4bc-70d9cb68e0c9-maas-api-tls podName:e79217db-5570-45db-b4bc-70d9cb68e0c9 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:27.308287885 +0000 UTC m=+2136.499482168 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e79217db-5570-45db-b4bc-70d9cb68e0c9-maas-api-tls") pod "maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" (UID: "e79217db-5570-45db-b4bc-70d9cb68e0c9") : secret "maas-api-e2e-inf-b-30704e-serving-cert" not found Jul 22 03:33:26.808821 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:26.808797 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e79217db-5570-45db-b4bc-70d9cb68e0c9-openshift-service-ca\") pod \"maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7\" (UID: \"e79217db-5570-45db-b4bc-70d9cb68e0c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" Jul 22 03:33:26.817347 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:26.817325 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fz6g\" (UniqueName: \"kubernetes.io/projected/e79217db-5570-45db-b4bc-70d9cb68e0c9-kube-api-access-2fz6g\") pod \"maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7\" (UID: \"e79217db-5570-45db-b4bc-70d9cb68e0c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" Jul 22 03:33:27.313982 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:27.313944 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e79217db-5570-45db-b4bc-70d9cb68e0c9-maas-api-tls\") pod \"maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7\" (UID: \"e79217db-5570-45db-b4bc-70d9cb68e0c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" Jul 22 03:33:27.316350 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:27.316321 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e79217db-5570-45db-b4bc-70d9cb68e0c9-maas-api-tls\") pod \"maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7\" (UID: \"e79217db-5570-45db-b4bc-70d9cb68e0c9\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" Jul 22 03:33:27.531656 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:27.531627 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" Jul 22 03:33:27.667154 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:27.667130 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7"] Jul 22 03:33:27.669764 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:33:27.669725 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode79217db_5570_45db_b4bc_70d9cb68e0c9.slice/crio-f209bac019511ac0880aff60316914cebdb2d8b899800a102ef06558baffef98 WatchSource:0}: Error finding container f209bac019511ac0880aff60316914cebdb2d8b899800a102ef06558baffef98: Status 404 returned error can't find the container with id f209bac019511ac0880aff60316914cebdb2d8b899800a102ef06558baffef98 Jul 22 03:33:28.343290 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:28.343207 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" event={"ID":"e79217db-5570-45db-b4bc-70d9cb68e0c9","Type":"ContainerStarted","Data":"28d654c14cc50a67c2f983708439c353c68cfbfbb84b8e3cec1609523dc28369"} Jul 22 03:33:28.343290 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:28.343256 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" event={"ID":"e79217db-5570-45db-b4bc-70d9cb68e0c9","Type":"ContainerStarted","Data":"f209bac019511ac0880aff60316914cebdb2d8b899800a102ef06558baffef98"} Jul 22 03:33:28.343787 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:28.343345 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" Jul 22 03:33:28.363740 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:28.363692 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" podStartSLOduration=2.035473289 podStartE2EDuration="2.36367874s" totalImagesPullingTime="328.205451ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:28.363594271 +0000 UTC m=+2137.554788574" watchObservedRunningTime="2026-07-22 03:33:28.36367874 +0000 UTC m=+2137.554873042" Jul 22 03:33:34.353122 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:33:34.353091 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" Jul 22 03:35:02.507326 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.507289 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm"] Jul 22 03:35:02.507884 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.507533 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" podUID="e282274b-0b60-4451-8ce2-a9ff9c6754c2" containerName="maas-api" containerID="cri-o://56a971c2ef596a04d0d215ade3825bc107d477f8fd227635ad0ff04b24976704" gracePeriod=30 Jul 22 03:35:02.760643 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.760571 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" Jul 22 03:35:02.793864 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.793802 2580 generic.go:356] "Generic (PLEG): container finished" podID="e282274b-0b60-4451-8ce2-a9ff9c6754c2" containerID="56a971c2ef596a04d0d215ade3825bc107d477f8fd227635ad0ff04b24976704" exitCode=0 Jul 22 03:35:02.794028 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.793872 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" Jul 22 03:35:02.794028 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.793888 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" event={"ID":"e282274b-0b60-4451-8ce2-a9ff9c6754c2","Type":"ContainerDied","Data":"56a971c2ef596a04d0d215ade3825bc107d477f8fd227635ad0ff04b24976704"} Jul 22 03:35:02.794028 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.793974 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm" event={"ID":"e282274b-0b60-4451-8ce2-a9ff9c6754c2","Type":"ContainerDied","Data":"d9b6e073cf3bff55541e056a3979c6ede31b43096fba16530969fdd038710147"} Jul 22 03:35:02.794028 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.794004 2580 scope.go:117] "RemoveContainer" containerID="56a971c2ef596a04d0d215ade3825bc107d477f8fd227635ad0ff04b24976704" Jul 22 03:35:02.802295 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.802278 2580 scope.go:117] "RemoveContainer" containerID="56a971c2ef596a04d0d215ade3825bc107d477f8fd227635ad0ff04b24976704" Jul 22 03:35:02.802521 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:35:02.802503 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56a971c2ef596a04d0d215ade3825bc107d477f8fd227635ad0ff04b24976704\": container with ID starting with 56a971c2ef596a04d0d215ade3825bc107d477f8fd227635ad0ff04b24976704 not found: ID does not exist" containerID="56a971c2ef596a04d0d215ade3825bc107d477f8fd227635ad0ff04b24976704" Jul 22 03:35:02.802560 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.802529 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56a971c2ef596a04d0d215ade3825bc107d477f8fd227635ad0ff04b24976704"} err="failed to get container status \"56a971c2ef596a04d0d215ade3825bc107d477f8fd227635ad0ff04b24976704\": rpc error: code = NotFound desc = could not find container \"56a971c2ef596a04d0d215ade3825bc107d477f8fd227635ad0ff04b24976704\": container with ID starting with 56a971c2ef596a04d0d215ade3825bc107d477f8fd227635ad0ff04b24976704 not found: ID does not exist" Jul 22 03:35:02.841583 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.841561 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkm5n\" (UniqueName: \"kubernetes.io/projected/e282274b-0b60-4451-8ce2-a9ff9c6754c2-kube-api-access-lkm5n\") pod \"e282274b-0b60-4451-8ce2-a9ff9c6754c2\" (UID: \"e282274b-0b60-4451-8ce2-a9ff9c6754c2\") " Jul 22 03:35:02.841679 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.841636 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e282274b-0b60-4451-8ce2-a9ff9c6754c2-openshift-service-ca\") pod \"e282274b-0b60-4451-8ce2-a9ff9c6754c2\" (UID: \"e282274b-0b60-4451-8ce2-a9ff9c6754c2\") " Jul 22 03:35:02.841679 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.841655 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e282274b-0b60-4451-8ce2-a9ff9c6754c2-maas-api-tls\") pod \"e282274b-0b60-4451-8ce2-a9ff9c6754c2\" (UID: \"e282274b-0b60-4451-8ce2-a9ff9c6754c2\") " Jul 22 03:35:02.841999 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.841966 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e282274b-0b60-4451-8ce2-a9ff9c6754c2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e282274b-0b60-4451-8ce2-a9ff9c6754c2" (UID: "e282274b-0b60-4451-8ce2-a9ff9c6754c2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:35:02.843650 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.843630 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e282274b-0b60-4451-8ce2-a9ff9c6754c2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e282274b-0b60-4451-8ce2-a9ff9c6754c2" (UID: "e282274b-0b60-4451-8ce2-a9ff9c6754c2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:35:02.843723 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.843700 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e282274b-0b60-4451-8ce2-a9ff9c6754c2-kube-api-access-lkm5n" (OuterVolumeSpecName: "kube-api-access-lkm5n") pod "e282274b-0b60-4451-8ce2-a9ff9c6754c2" (UID: "e282274b-0b60-4451-8ce2-a9ff9c6754c2"). InnerVolumeSpecName "kube-api-access-lkm5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:35:02.942441 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.942413 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e282274b-0b60-4451-8ce2-a9ff9c6754c2-openshift-service-ca\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:35:02.942441 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.942439 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e282274b-0b60-4451-8ce2-a9ff9c6754c2-maas-api-tls\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:35:02.942441 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:02.942448 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkm5n\" (UniqueName: \"kubernetes.io/projected/e282274b-0b60-4451-8ce2-a9ff9c6754c2-kube-api-access-lkm5n\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:35:03.115661 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:03.115634 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm"] Jul 22 03:35:03.119566 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:03.119541 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fd4878-6597c6c755-2z4nm"] Jul 22 03:35:03.368282 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:03.368209 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e282274b-0b60-4451-8ce2-a9ff9c6754c2" path="/var/lib/kubelet/pods/e282274b-0b60-4451-8ce2-a9ff9c6754c2/volumes" Jul 22 03:35:24.873968 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:24.873878 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7"] Jul 22 03:35:24.874447 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:24.874129 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" podUID="e79217db-5570-45db-b4bc-70d9cb68e0c9" containerName="maas-api" containerID="cri-o://28d654c14cc50a67c2f983708439c353c68cfbfbb84b8e3cec1609523dc28369" gracePeriod=30 Jul 22 03:35:25.127340 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.127284 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" Jul 22 03:35:25.141869 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.141820 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fz6g\" (UniqueName: \"kubernetes.io/projected/e79217db-5570-45db-b4bc-70d9cb68e0c9-kube-api-access-2fz6g\") pod \"e79217db-5570-45db-b4bc-70d9cb68e0c9\" (UID: \"e79217db-5570-45db-b4bc-70d9cb68e0c9\") " Jul 22 03:35:25.141992 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.141891 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e79217db-5570-45db-b4bc-70d9cb68e0c9-openshift-service-ca\") pod \"e79217db-5570-45db-b4bc-70d9cb68e0c9\" (UID: \"e79217db-5570-45db-b4bc-70d9cb68e0c9\") " Jul 22 03:35:25.142037 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.141990 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e79217db-5570-45db-b4bc-70d9cb68e0c9-maas-api-tls\") pod \"e79217db-5570-45db-b4bc-70d9cb68e0c9\" (UID: \"e79217db-5570-45db-b4bc-70d9cb68e0c9\") " Jul 22 03:35:25.142285 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.142249 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e79217db-5570-45db-b4bc-70d9cb68e0c9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e79217db-5570-45db-b4bc-70d9cb68e0c9" (UID: "e79217db-5570-45db-b4bc-70d9cb68e0c9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:35:25.143790 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.143766 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e79217db-5570-45db-b4bc-70d9cb68e0c9-kube-api-access-2fz6g" (OuterVolumeSpecName: "kube-api-access-2fz6g") pod "e79217db-5570-45db-b4bc-70d9cb68e0c9" (UID: "e79217db-5570-45db-b4bc-70d9cb68e0c9"). InnerVolumeSpecName "kube-api-access-2fz6g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:35:25.144005 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.143967 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e79217db-5570-45db-b4bc-70d9cb68e0c9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e79217db-5570-45db-b4bc-70d9cb68e0c9" (UID: "e79217db-5570-45db-b4bc-70d9cb68e0c9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:35:25.243637 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.243612 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e79217db-5570-45db-b4bc-70d9cb68e0c9-openshift-service-ca\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:35:25.243637 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.243636 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e79217db-5570-45db-b4bc-70d9cb68e0c9-maas-api-tls\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:35:25.243803 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.243645 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fz6g\" (UniqueName: \"kubernetes.io/projected/e79217db-5570-45db-b4bc-70d9cb68e0c9-kube-api-access-2fz6g\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:35:25.910810 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.910776 2580 generic.go:356] "Generic (PLEG): container finished" podID="e79217db-5570-45db-b4bc-70d9cb68e0c9" containerID="28d654c14cc50a67c2f983708439c353c68cfbfbb84b8e3cec1609523dc28369" exitCode=0 Jul 22 03:35:25.911271 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.910819 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" Jul 22 03:35:25.911271 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.910826 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" event={"ID":"e79217db-5570-45db-b4bc-70d9cb68e0c9","Type":"ContainerDied","Data":"28d654c14cc50a67c2f983708439c353c68cfbfbb84b8e3cec1609523dc28369"} Jul 22 03:35:25.911271 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.910910 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7" event={"ID":"e79217db-5570-45db-b4bc-70d9cb68e0c9","Type":"ContainerDied","Data":"f209bac019511ac0880aff60316914cebdb2d8b899800a102ef06558baffef98"} Jul 22 03:35:25.911271 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.910944 2580 scope.go:117] "RemoveContainer" containerID="28d654c14cc50a67c2f983708439c353c68cfbfbb84b8e3cec1609523dc28369" Jul 22 03:35:25.919231 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.919216 2580 scope.go:117] "RemoveContainer" containerID="28d654c14cc50a67c2f983708439c353c68cfbfbb84b8e3cec1609523dc28369" Jul 22 03:35:25.919489 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:35:25.919461 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28d654c14cc50a67c2f983708439c353c68cfbfbb84b8e3cec1609523dc28369\": container with ID starting with 28d654c14cc50a67c2f983708439c353c68cfbfbb84b8e3cec1609523dc28369 not found: ID does not exist" containerID="28d654c14cc50a67c2f983708439c353c68cfbfbb84b8e3cec1609523dc28369" Jul 22 03:35:25.919534 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.919497 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28d654c14cc50a67c2f983708439c353c68cfbfbb84b8e3cec1609523dc28369"} err="failed to get container status \"28d654c14cc50a67c2f983708439c353c68cfbfbb84b8e3cec1609523dc28369\": rpc error: code = NotFound desc = could not find container \"28d654c14cc50a67c2f983708439c353c68cfbfbb84b8e3cec1609523dc28369\": container with ID starting with 28d654c14cc50a67c2f983708439c353c68cfbfbb84b8e3cec1609523dc28369 not found: ID does not exist" Jul 22 03:35:25.927111 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.927090 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7"] Jul 22 03:35:25.931040 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:25.931021 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-30704e-cbb685b7d-t7lc7"] Jul 22 03:35:27.366567 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:27.366535 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e79217db-5570-45db-b4bc-70d9cb68e0c9" path="/var/lib/kubelet/pods/e79217db-5570-45db-b4bc-70d9cb68e0c9/volumes" Jul 22 03:35:40.210922 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.210889 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb"] Jul 22 03:35:40.211457 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.211422 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e79217db-5570-45db-b4bc-70d9cb68e0c9" containerName="maas-api" Jul 22 03:35:40.211457 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.211437 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e79217db-5570-45db-b4bc-70d9cb68e0c9" containerName="maas-api" Jul 22 03:35:40.211584 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.211461 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e282274b-0b60-4451-8ce2-a9ff9c6754c2" containerName="maas-api" Jul 22 03:35:40.211584 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.211469 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e282274b-0b60-4451-8ce2-a9ff9c6754c2" containerName="maas-api" Jul 22 03:35:40.211584 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.211553 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e79217db-5570-45db-b4bc-70d9cb68e0c9" containerName="maas-api" Jul 22 03:35:40.211584 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.211567 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e282274b-0b60-4451-8ce2-a9ff9c6754c2" containerName="maas-api" Jul 22 03:35:40.214422 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.214398 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" Jul 22 03:35:40.216659 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.216634 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-19cc62-serving-cert\"" Jul 22 03:35:40.225180 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.225154 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb"] Jul 22 03:35:40.276561 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.276532 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cf89ce5-c68d-4c44-8aba-b6eabe016234-maas-api-tls\") pod \"maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb\" (UID: \"7cf89ce5-c68d-4c44-8aba-b6eabe016234\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" Jul 22 03:35:40.276702 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.276572 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shzvr\" (UniqueName: \"kubernetes.io/projected/7cf89ce5-c68d-4c44-8aba-b6eabe016234-kube-api-access-shzvr\") pod \"maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb\" (UID: \"7cf89ce5-c68d-4c44-8aba-b6eabe016234\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" Jul 22 03:35:40.276702 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.276620 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cf89ce5-c68d-4c44-8aba-b6eabe016234-openshift-service-ca\") pod \"maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb\" (UID: \"7cf89ce5-c68d-4c44-8aba-b6eabe016234\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" Jul 22 03:35:40.377718 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.377692 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cf89ce5-c68d-4c44-8aba-b6eabe016234-maas-api-tls\") pod \"maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb\" (UID: \"7cf89ce5-c68d-4c44-8aba-b6eabe016234\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" Jul 22 03:35:40.377883 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.377732 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shzvr\" (UniqueName: \"kubernetes.io/projected/7cf89ce5-c68d-4c44-8aba-b6eabe016234-kube-api-access-shzvr\") pod \"maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb\" (UID: \"7cf89ce5-c68d-4c44-8aba-b6eabe016234\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" Jul 22 03:35:40.377883 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:35:40.377839 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-serving-cert: secret "maas-api-e2e-api-a-19cc62-serving-cert" not found Jul 22 03:35:40.378003 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.377880 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cf89ce5-c68d-4c44-8aba-b6eabe016234-openshift-service-ca\") pod \"maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb\" (UID: \"7cf89ce5-c68d-4c44-8aba-b6eabe016234\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" Jul 22 03:35:40.378003 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:35:40.377892 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cf89ce5-c68d-4c44-8aba-b6eabe016234-maas-api-tls podName:7cf89ce5-c68d-4c44-8aba-b6eabe016234 nodeName:}" failed. No retries permitted until 2026-07-22 03:35:40.877876557 +0000 UTC m=+2270.069070838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7cf89ce5-c68d-4c44-8aba-b6eabe016234-maas-api-tls") pod "maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" (UID: "7cf89ce5-c68d-4c44-8aba-b6eabe016234") : secret "maas-api-e2e-api-a-19cc62-serving-cert" not found Jul 22 03:35:40.378447 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.378428 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cf89ce5-c68d-4c44-8aba-b6eabe016234-openshift-service-ca\") pod \"maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb\" (UID: \"7cf89ce5-c68d-4c44-8aba-b6eabe016234\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" Jul 22 03:35:40.391242 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.391222 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shzvr\" (UniqueName: \"kubernetes.io/projected/7cf89ce5-c68d-4c44-8aba-b6eabe016234-kube-api-access-shzvr\") pod \"maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb\" (UID: \"7cf89ce5-c68d-4c44-8aba-b6eabe016234\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" Jul 22 03:35:40.882774 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.882740 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cf89ce5-c68d-4c44-8aba-b6eabe016234-maas-api-tls\") pod \"maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb\" (UID: \"7cf89ce5-c68d-4c44-8aba-b6eabe016234\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" Jul 22 03:35:40.885255 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:40.885231 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cf89ce5-c68d-4c44-8aba-b6eabe016234-maas-api-tls\") pod \"maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb\" (UID: \"7cf89ce5-c68d-4c44-8aba-b6eabe016234\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" Jul 22 03:35:41.126607 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:41.126571 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" Jul 22 03:35:41.463346 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:41.463324 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb"] Jul 22 03:35:41.472060 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:35:41.472023 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cf89ce5_c68d_4c44_8aba_b6eabe016234.slice/crio-722e84141e138f7f96de8c1350faddf3093754225e763496129bf1e70db248bc WatchSource:0}: Error finding container 722e84141e138f7f96de8c1350faddf3093754225e763496129bf1e70db248bc: Status 404 returned error can't find the container with id 722e84141e138f7f96de8c1350faddf3093754225e763496129bf1e70db248bc Jul 22 03:35:41.473757 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:41.473736 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 03:35:41.986960 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:41.986931 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" event={"ID":"7cf89ce5-c68d-4c44-8aba-b6eabe016234","Type":"ContainerStarted","Data":"f32736caf85396050a77d6a03a1a320b37420aeaff72bc76d65185297f7ec8b0"} Jul 22 03:35:41.987071 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:41.987028 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" Jul 22 03:35:41.987071 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:41.987055 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" event={"ID":"7cf89ce5-c68d-4c44-8aba-b6eabe016234","Type":"ContainerStarted","Data":"722e84141e138f7f96de8c1350faddf3093754225e763496129bf1e70db248bc"} Jul 22 03:35:42.005972 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:42.005925 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" podStartSLOduration=1.5819133829999998 podStartE2EDuration="2.005910355s" totalImagesPullingTime="423.996972ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:35:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:35:42.00420558 +0000 UTC m=+2271.195399883" watchObservedRunningTime="2026-07-22 03:35:42.005910355 +0000 UTC m=+2271.197104656" Jul 22 03:35:47.997064 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:47.997035 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" Jul 22 03:35:48.263643 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:48.260976 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr"] Jul 22 03:35:48.266585 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:48.266559 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" Jul 22 03:35:48.269057 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:48.269040 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-ed08a5-serving-cert\"" Jul 22 03:35:48.275557 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:48.275534 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr"] Jul 22 03:35:48.344464 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:48.344427 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjsft\" (UniqueName: \"kubernetes.io/projected/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-kube-api-access-zjsft\") pod \"maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr\" (UID: \"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" Jul 22 03:35:48.344629 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:48.344554 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-maas-api-tls\") pod \"maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr\" (UID: \"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" Jul 22 03:35:48.344695 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:48.344640 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-openshift-service-ca\") pod \"maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr\" (UID: \"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" Jul 22 03:35:48.446076 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:48.446043 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-maas-api-tls\") pod \"maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr\" (UID: \"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" Jul 22 03:35:48.446232 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:48.446111 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-openshift-service-ca\") pod \"maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr\" (UID: \"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" Jul 22 03:35:48.446232 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:48.446143 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjsft\" (UniqueName: \"kubernetes.io/projected/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-kube-api-access-zjsft\") pod \"maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr\" (UID: \"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" Jul 22 03:35:48.446232 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:35:48.446197 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-serving-cert: secret "maas-api-e2e-api-b-ed08a5-serving-cert" not found Jul 22 03:35:48.446392 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:35:48.446271 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-maas-api-tls podName:cc157521-e5b4-4fca-b2e3-a6c74f8f9a13 nodeName:}" failed. No retries permitted until 2026-07-22 03:35:48.946251973 +0000 UTC m=+2278.137446257 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-maas-api-tls") pod "maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" (UID: "cc157521-e5b4-4fca-b2e3-a6c74f8f9a13") : secret "maas-api-e2e-api-b-ed08a5-serving-cert" not found Jul 22 03:35:48.446785 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:48.446762 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-openshift-service-ca\") pod \"maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr\" (UID: \"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" Jul 22 03:35:48.456081 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:48.456056 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjsft\" (UniqueName: \"kubernetes.io/projected/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-kube-api-access-zjsft\") pod \"maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr\" (UID: \"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" Jul 22 03:35:48.950809 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:48.950771 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-maas-api-tls\") pod \"maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr\" (UID: \"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" Jul 22 03:35:48.953062 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:48.953034 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-maas-api-tls\") pod \"maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr\" (UID: \"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" Jul 22 03:35:49.178271 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:49.178238 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" Jul 22 03:35:49.320480 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:49.320456 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr"] Jul 22 03:35:49.323249 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:35:49.323224 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc157521_e5b4_4fca_b2e3_a6c74f8f9a13.slice/crio-938cb51b2270484d1882a2b7e832f2abfcd22abc956693d307abdb82cfca5663 WatchSource:0}: Error finding container 938cb51b2270484d1882a2b7e832f2abfcd22abc956693d307abdb82cfca5663: Status 404 returned error can't find the container with id 938cb51b2270484d1882a2b7e832f2abfcd22abc956693d307abdb82cfca5663 Jul 22 03:35:50.027869 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:50.027762 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" event={"ID":"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13","Type":"ContainerStarted","Data":"bebd0f574187dc2cd2a5c41920ac81f899d06c48ac881f40832ce0a444bb2a2c"} Jul 22 03:35:50.027869 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:50.027815 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" event={"ID":"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13","Type":"ContainerStarted","Data":"938cb51b2270484d1882a2b7e832f2abfcd22abc956693d307abdb82cfca5663"} Jul 22 03:35:50.028048 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:50.027877 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" Jul 22 03:35:50.053163 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:50.053112 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" podStartSLOduration=1.676280678 podStartE2EDuration="2.053093087s" totalImagesPullingTime="376.812409ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:35:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:35:50.051474137 +0000 UTC m=+2279.242668443" watchObservedRunningTime="2026-07-22 03:35:50.053093087 +0000 UTC m=+2279.244287390" Jul 22 03:35:56.037615 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:35:56.037582 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" Jul 22 03:36:16.775114 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:16.775085 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb"] Jul 22 03:36:16.775649 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:16.775345 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" podUID="7cf89ce5-c68d-4c44-8aba-b6eabe016234" containerName="maas-api" containerID="cri-o://f32736caf85396050a77d6a03a1a320b37420aeaff72bc76d65185297f7ec8b0" gracePeriod=30 Jul 22 03:36:17.027424 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.027368 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" Jul 22 03:36:17.108595 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.108565 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cf89ce5-c68d-4c44-8aba-b6eabe016234-openshift-service-ca\") pod \"7cf89ce5-c68d-4c44-8aba-b6eabe016234\" (UID: \"7cf89ce5-c68d-4c44-8aba-b6eabe016234\") " Jul 22 03:36:17.108774 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.108694 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shzvr\" (UniqueName: \"kubernetes.io/projected/7cf89ce5-c68d-4c44-8aba-b6eabe016234-kube-api-access-shzvr\") pod \"7cf89ce5-c68d-4c44-8aba-b6eabe016234\" (UID: \"7cf89ce5-c68d-4c44-8aba-b6eabe016234\") " Jul 22 03:36:17.108774 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.108721 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cf89ce5-c68d-4c44-8aba-b6eabe016234-maas-api-tls\") pod \"7cf89ce5-c68d-4c44-8aba-b6eabe016234\" (UID: \"7cf89ce5-c68d-4c44-8aba-b6eabe016234\") " Jul 22 03:36:17.109029 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.108997 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cf89ce5-c68d-4c44-8aba-b6eabe016234-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7cf89ce5-c68d-4c44-8aba-b6eabe016234" (UID: "7cf89ce5-c68d-4c44-8aba-b6eabe016234"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:36:17.110884 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.110858 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cf89ce5-c68d-4c44-8aba-b6eabe016234-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7cf89ce5-c68d-4c44-8aba-b6eabe016234" (UID: "7cf89ce5-c68d-4c44-8aba-b6eabe016234"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:36:17.110977 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.110920 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cf89ce5-c68d-4c44-8aba-b6eabe016234-kube-api-access-shzvr" (OuterVolumeSpecName: "kube-api-access-shzvr") pod "7cf89ce5-c68d-4c44-8aba-b6eabe016234" (UID: "7cf89ce5-c68d-4c44-8aba-b6eabe016234"). InnerVolumeSpecName "kube-api-access-shzvr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:36:17.161477 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.161448 2580 generic.go:356] "Generic (PLEG): container finished" podID="7cf89ce5-c68d-4c44-8aba-b6eabe016234" containerID="f32736caf85396050a77d6a03a1a320b37420aeaff72bc76d65185297f7ec8b0" exitCode=0 Jul 22 03:36:17.161572 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.161493 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" event={"ID":"7cf89ce5-c68d-4c44-8aba-b6eabe016234","Type":"ContainerDied","Data":"f32736caf85396050a77d6a03a1a320b37420aeaff72bc76d65185297f7ec8b0"} Jul 22 03:36:17.161572 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.161529 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" event={"ID":"7cf89ce5-c68d-4c44-8aba-b6eabe016234","Type":"ContainerDied","Data":"722e84141e138f7f96de8c1350faddf3093754225e763496129bf1e70db248bc"} Jul 22 03:36:17.161572 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.161549 2580 scope.go:117] "RemoveContainer" containerID="f32736caf85396050a77d6a03a1a320b37420aeaff72bc76d65185297f7ec8b0" Jul 22 03:36:17.162086 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.161494 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb" Jul 22 03:36:17.169725 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.169709 2580 scope.go:117] "RemoveContainer" containerID="f32736caf85396050a77d6a03a1a320b37420aeaff72bc76d65185297f7ec8b0" Jul 22 03:36:17.169978 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:36:17.169960 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f32736caf85396050a77d6a03a1a320b37420aeaff72bc76d65185297f7ec8b0\": container with ID starting with f32736caf85396050a77d6a03a1a320b37420aeaff72bc76d65185297f7ec8b0 not found: ID does not exist" containerID="f32736caf85396050a77d6a03a1a320b37420aeaff72bc76d65185297f7ec8b0" Jul 22 03:36:17.170050 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.169989 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f32736caf85396050a77d6a03a1a320b37420aeaff72bc76d65185297f7ec8b0"} err="failed to get container status \"f32736caf85396050a77d6a03a1a320b37420aeaff72bc76d65185297f7ec8b0\": rpc error: code = NotFound desc = could not find container \"f32736caf85396050a77d6a03a1a320b37420aeaff72bc76d65185297f7ec8b0\": container with ID starting with f32736caf85396050a77d6a03a1a320b37420aeaff72bc76d65185297f7ec8b0 not found: ID does not exist" Jul 22 03:36:17.190960 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.190775 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb"] Jul 22 03:36:17.192884 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.192866 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-19cc62-76fb45dc67-w4wdb"] Jul 22 03:36:17.209904 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.209882 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shzvr\" (UniqueName: \"kubernetes.io/projected/7cf89ce5-c68d-4c44-8aba-b6eabe016234-kube-api-access-shzvr\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:36:17.209904 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.209903 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cf89ce5-c68d-4c44-8aba-b6eabe016234-maas-api-tls\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:36:17.210034 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.209917 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cf89ce5-c68d-4c44-8aba-b6eabe016234-openshift-service-ca\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:36:17.366390 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:17.366354 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cf89ce5-c68d-4c44-8aba-b6eabe016234" path="/var/lib/kubelet/pods/7cf89ce5-c68d-4c44-8aba-b6eabe016234/volumes" Jul 22 03:36:43.777294 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:43.777220 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr"] Jul 22 03:36:43.777669 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:43.777447 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" podUID="cc157521-e5b4-4fca-b2e3-a6c74f8f9a13" containerName="maas-api" containerID="cri-o://bebd0f574187dc2cd2a5c41920ac81f899d06c48ac881f40832ce0a444bb2a2c" gracePeriod=30 Jul 22 03:36:44.019773 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.019749 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" Jul 22 03:36:44.139097 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.139069 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-maas-api-tls\") pod \"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13\" (UID: \"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13\") " Jul 22 03:36:44.139267 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.139107 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjsft\" (UniqueName: \"kubernetes.io/projected/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-kube-api-access-zjsft\") pod \"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13\" (UID: \"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13\") " Jul 22 03:36:44.139267 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.139156 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-openshift-service-ca\") pod \"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13\" (UID: \"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13\") " Jul 22 03:36:44.139500 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.139470 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cc157521-e5b4-4fca-b2e3-a6c74f8f9a13" (UID: "cc157521-e5b4-4fca-b2e3-a6c74f8f9a13"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:36:44.141138 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.141117 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cc157521-e5b4-4fca-b2e3-a6c74f8f9a13" (UID: "cc157521-e5b4-4fca-b2e3-a6c74f8f9a13"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:36:44.141218 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.141168 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-kube-api-access-zjsft" (OuterVolumeSpecName: "kube-api-access-zjsft") pod "cc157521-e5b4-4fca-b2e3-a6c74f8f9a13" (UID: "cc157521-e5b4-4fca-b2e3-a6c74f8f9a13"). InnerVolumeSpecName "kube-api-access-zjsft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:36:44.240011 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.239984 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-maas-api-tls\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:36:44.240011 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.240007 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjsft\" (UniqueName: \"kubernetes.io/projected/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-kube-api-access-zjsft\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:36:44.240011 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.240017 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13-openshift-service-ca\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:36:44.286705 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.286675 2580 generic.go:356] "Generic (PLEG): container finished" podID="cc157521-e5b4-4fca-b2e3-a6c74f8f9a13" containerID="bebd0f574187dc2cd2a5c41920ac81f899d06c48ac881f40832ce0a444bb2a2c" exitCode=0 Jul 22 03:36:44.286814 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.286716 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" Jul 22 03:36:44.286814 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.286743 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" event={"ID":"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13","Type":"ContainerDied","Data":"bebd0f574187dc2cd2a5c41920ac81f899d06c48ac881f40832ce0a444bb2a2c"} Jul 22 03:36:44.286814 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.286791 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr" event={"ID":"cc157521-e5b4-4fca-b2e3-a6c74f8f9a13","Type":"ContainerDied","Data":"938cb51b2270484d1882a2b7e832f2abfcd22abc956693d307abdb82cfca5663"} Jul 22 03:36:44.286942 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.286816 2580 scope.go:117] "RemoveContainer" containerID="bebd0f574187dc2cd2a5c41920ac81f899d06c48ac881f40832ce0a444bb2a2c" Jul 22 03:36:44.295325 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.295304 2580 scope.go:117] "RemoveContainer" containerID="bebd0f574187dc2cd2a5c41920ac81f899d06c48ac881f40832ce0a444bb2a2c" Jul 22 03:36:44.295554 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:36:44.295536 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bebd0f574187dc2cd2a5c41920ac81f899d06c48ac881f40832ce0a444bb2a2c\": container with ID starting with bebd0f574187dc2cd2a5c41920ac81f899d06c48ac881f40832ce0a444bb2a2c not found: ID does not exist" containerID="bebd0f574187dc2cd2a5c41920ac81f899d06c48ac881f40832ce0a444bb2a2c" Jul 22 03:36:44.295603 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.295562 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bebd0f574187dc2cd2a5c41920ac81f899d06c48ac881f40832ce0a444bb2a2c"} err="failed to get container status \"bebd0f574187dc2cd2a5c41920ac81f899d06c48ac881f40832ce0a444bb2a2c\": rpc error: code = NotFound desc = could not find container \"bebd0f574187dc2cd2a5c41920ac81f899d06c48ac881f40832ce0a444bb2a2c\": container with ID starting with bebd0f574187dc2cd2a5c41920ac81f899d06c48ac881f40832ce0a444bb2a2c not found: ID does not exist" Jul 22 03:36:44.309073 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.309050 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr"] Jul 22 03:36:44.315252 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:44.315225 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-ed08a5-54ddc9959f-xjldr"] Jul 22 03:36:45.366985 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:45.366949 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc157521-e5b4-4fca-b2e3-a6c74f8f9a13" path="/var/lib/kubelet/pods/cc157521-e5b4-4fca-b2e3-a6c74f8f9a13/volumes" Jul 22 03:36:59.435888 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.435857 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh"] Jul 22 03:36:59.436407 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.436388 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cf89ce5-c68d-4c44-8aba-b6eabe016234" containerName="maas-api" Jul 22 03:36:59.436489 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.436410 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cf89ce5-c68d-4c44-8aba-b6eabe016234" containerName="maas-api" Jul 22 03:36:59.436489 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.436425 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc157521-e5b4-4fca-b2e3-a6c74f8f9a13" containerName="maas-api" Jul 22 03:36:59.436489 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.436433 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc157521-e5b4-4fca-b2e3-a6c74f8f9a13" containerName="maas-api" Jul 22 03:36:59.436633 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.436528 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cf89ce5-c68d-4c44-8aba-b6eabe016234" containerName="maas-api" Jul 22 03:36:59.436633 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.436541 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc157521-e5b4-4fca-b2e3-a6c74f8f9a13" containerName="maas-api" Jul 22 03:36:59.439485 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.439464 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" Jul 22 03:36:59.442002 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.441973 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-c19e4b-serving-cert\"" Jul 22 03:36:59.451819 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.451793 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh"] Jul 22 03:36:59.483611 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.483586 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hkxz\" (UniqueName: \"kubernetes.io/projected/b00beb26-7214-4b13-9734-b440d33f1ad2-kube-api-access-2hkxz\") pod \"maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh\" (UID: \"b00beb26-7214-4b13-9734-b440d33f1ad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" Jul 22 03:36:59.483748 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.483630 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b00beb26-7214-4b13-9734-b440d33f1ad2-maas-api-tls\") pod \"maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh\" (UID: \"b00beb26-7214-4b13-9734-b440d33f1ad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" Jul 22 03:36:59.483748 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.483655 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b00beb26-7214-4b13-9734-b440d33f1ad2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh\" (UID: \"b00beb26-7214-4b13-9734-b440d33f1ad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" Jul 22 03:36:59.585197 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.585167 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hkxz\" (UniqueName: \"kubernetes.io/projected/b00beb26-7214-4b13-9734-b440d33f1ad2-kube-api-access-2hkxz\") pod \"maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh\" (UID: \"b00beb26-7214-4b13-9734-b440d33f1ad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" Jul 22 03:36:59.585354 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.585208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b00beb26-7214-4b13-9734-b440d33f1ad2-maas-api-tls\") pod \"maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh\" (UID: \"b00beb26-7214-4b13-9734-b440d33f1ad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" Jul 22 03:36:59.585354 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.585241 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b00beb26-7214-4b13-9734-b440d33f1ad2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh\" (UID: \"b00beb26-7214-4b13-9734-b440d33f1ad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" Jul 22 03:36:59.585354 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:36:59.585339 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-serving-cert: secret "maas-api-e2e-shared-a-c19e4b-serving-cert" not found Jul 22 03:36:59.585525 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:36:59.585426 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b00beb26-7214-4b13-9734-b440d33f1ad2-maas-api-tls podName:b00beb26-7214-4b13-9734-b440d33f1ad2 nodeName:}" failed. No retries permitted until 2026-07-22 03:37:00.085409207 +0000 UTC m=+2349.276603488 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b00beb26-7214-4b13-9734-b440d33f1ad2-maas-api-tls") pod "maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" (UID: "b00beb26-7214-4b13-9734-b440d33f1ad2") : secret "maas-api-e2e-shared-a-c19e4b-serving-cert" not found Jul 22 03:36:59.585954 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.585934 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b00beb26-7214-4b13-9734-b440d33f1ad2-openshift-service-ca\") pod \"maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh\" (UID: \"b00beb26-7214-4b13-9734-b440d33f1ad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" Jul 22 03:36:59.597536 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:36:59.597511 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hkxz\" (UniqueName: \"kubernetes.io/projected/b00beb26-7214-4b13-9734-b440d33f1ad2-kube-api-access-2hkxz\") pod \"maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh\" (UID: \"b00beb26-7214-4b13-9734-b440d33f1ad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" Jul 22 03:37:00.090778 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:00.090744 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b00beb26-7214-4b13-9734-b440d33f1ad2-maas-api-tls\") pod \"maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh\" (UID: \"b00beb26-7214-4b13-9734-b440d33f1ad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" Jul 22 03:37:00.093267 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:00.093245 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b00beb26-7214-4b13-9734-b440d33f1ad2-maas-api-tls\") pod \"maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh\" (UID: \"b00beb26-7214-4b13-9734-b440d33f1ad2\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" Jul 22 03:37:00.350053 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:00.350022 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" Jul 22 03:37:00.487979 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:00.487956 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh"] Jul 22 03:37:00.489925 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:37:00.489898 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb00beb26_7214_4b13_9734_b440d33f1ad2.slice/crio-99bdc20f770c8f2cac68622efe621389ef6d70a89a2eb9335ac4978b3d294d2e WatchSource:0}: Error finding container 99bdc20f770c8f2cac68622efe621389ef6d70a89a2eb9335ac4978b3d294d2e: Status 404 returned error can't find the container with id 99bdc20f770c8f2cac68622efe621389ef6d70a89a2eb9335ac4978b3d294d2e Jul 22 03:37:01.376610 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:01.376570 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" event={"ID":"b00beb26-7214-4b13-9734-b440d33f1ad2","Type":"ContainerStarted","Data":"b5af0f8f55526f3878a3ada2104ed303d34ef2d0ddf38bcfb4253829b2744178"} Jul 22 03:37:01.376801 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:01.376642 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" event={"ID":"b00beb26-7214-4b13-9734-b440d33f1ad2","Type":"ContainerStarted","Data":"99bdc20f770c8f2cac68622efe621389ef6d70a89a2eb9335ac4978b3d294d2e"} Jul 22 03:37:01.376801 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:01.376663 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" Jul 22 03:37:01.396338 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:01.396289 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" podStartSLOduration=1.972035123 podStartE2EDuration="2.396274635s" totalImagesPullingTime="424.239512ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:36:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:37:01.395360767 +0000 UTC m=+2350.586555096" watchObservedRunningTime="2026-07-22 03:37:01.396274635 +0000 UTC m=+2350.587468936" Jul 22 03:37:07.386954 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:07.386924 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" Jul 22 03:37:07.507028 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:07.506989 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc"] Jul 22 03:37:07.510642 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:07.510619 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" Jul 22 03:37:07.512733 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:07.512710 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-49462a-serving-cert\"" Jul 22 03:37:07.525791 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:07.525766 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc"] Jul 22 03:37:07.558839 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:07.558791 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-maas-api-tls\") pod \"maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc\" (UID: \"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" Jul 22 03:37:07.559004 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:07.558897 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc\" (UID: \"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" Jul 22 03:37:07.559004 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:07.558929 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjhk6\" (UniqueName: \"kubernetes.io/projected/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-kube-api-access-rjhk6\") pod \"maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc\" (UID: \"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" Jul 22 03:37:07.659628 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:07.659540 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc\" (UID: \"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" Jul 22 03:37:07.659628 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:07.659587 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjhk6\" (UniqueName: \"kubernetes.io/projected/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-kube-api-access-rjhk6\") pod \"maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc\" (UID: \"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" Jul 22 03:37:07.659846 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:07.659668 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-maas-api-tls\") pod \"maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc\" (UID: \"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" Jul 22 03:37:07.659846 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:37:07.659781 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-serving-cert: secret "maas-api-e2e-shared-b-49462a-serving-cert" not found Jul 22 03:37:07.659925 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:37:07.659870 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-maas-api-tls podName:f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3 nodeName:}" failed. No retries permitted until 2026-07-22 03:37:08.159847025 +0000 UTC m=+2357.351041312 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-maas-api-tls") pod "maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" (UID: "f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3") : secret "maas-api-e2e-shared-b-49462a-serving-cert" not found Jul 22 03:37:07.660204 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:07.660183 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc\" (UID: \"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" Jul 22 03:37:07.670235 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:07.670201 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjhk6\" (UniqueName: \"kubernetes.io/projected/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-kube-api-access-rjhk6\") pod \"maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc\" (UID: \"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" Jul 22 03:37:08.165441 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:08.165407 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-maas-api-tls\") pod \"maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc\" (UID: \"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" Jul 22 03:37:08.167806 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:08.167782 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-maas-api-tls\") pod \"maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc\" (UID: \"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" Jul 22 03:37:08.424498 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:08.424407 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" Jul 22 03:37:08.558984 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:08.558951 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc"] Jul 22 03:37:08.561538 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:37:08.561497 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf83ab8eb_eab5_4f61_8cc5_1c0b5ebf5de3.slice/crio-be5cc48fb779444cc064522e8ca28e2cd254c21b68e9d749aff0b326168469b4 WatchSource:0}: Error finding container be5cc48fb779444cc064522e8ca28e2cd254c21b68e9d749aff0b326168469b4: Status 404 returned error can't find the container with id be5cc48fb779444cc064522e8ca28e2cd254c21b68e9d749aff0b326168469b4 Jul 22 03:37:09.415082 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:09.415035 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" event={"ID":"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3","Type":"ContainerStarted","Data":"3c38cfc875164525d16b1a0c26236f5f43fbef6bf9d30c047f575877c7be9306"} Jul 22 03:37:09.415261 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:09.415089 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" event={"ID":"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3","Type":"ContainerStarted","Data":"be5cc48fb779444cc064522e8ca28e2cd254c21b68e9d749aff0b326168469b4"} Jul 22 03:37:09.415261 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:09.415160 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" Jul 22 03:37:09.452086 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:09.452023 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" podStartSLOduration=1.7586799050000002 podStartE2EDuration="2.452000822s" totalImagesPullingTime="693.320917ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:37:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:37:09.449916522 +0000 UTC m=+2358.641110824" watchObservedRunningTime="2026-07-22 03:37:09.452000822 +0000 UTC m=+2358.643195127" Jul 22 03:37:15.426366 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:15.426336 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" Jul 22 03:37:51.430207 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:51.430171 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:37:51.436453 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:51.436429 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:37:51.436584 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:51.436503 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:37:51.442527 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:37:51.442507 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:42:51.458878 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:42:51.458776 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:42:51.465628 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:42:51.465604 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:42:51.466073 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:42:51.466058 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:42:51.472353 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:42:51.472336 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-zptvl_e9a37af0-c577-48af-9af1-c6089fb43ac2/ovn-acl-logging/0.log" Jul 22 03:43:41.679660 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:41.679625 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh"] Jul 22 03:43:41.680191 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:41.679905 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" podUID="b00beb26-7214-4b13-9734-b440d33f1ad2" containerName="maas-api" containerID="cri-o://b5af0f8f55526f3878a3ada2104ed303d34ef2d0ddf38bcfb4253829b2744178" gracePeriod=30 Jul 22 03:43:41.925788 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:41.925762 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" Jul 22 03:43:42.043064 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.042980 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b00beb26-7214-4b13-9734-b440d33f1ad2-maas-api-tls\") pod \"b00beb26-7214-4b13-9734-b440d33f1ad2\" (UID: \"b00beb26-7214-4b13-9734-b440d33f1ad2\") " Jul 22 03:43:42.043064 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.043021 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hkxz\" (UniqueName: \"kubernetes.io/projected/b00beb26-7214-4b13-9734-b440d33f1ad2-kube-api-access-2hkxz\") pod \"b00beb26-7214-4b13-9734-b440d33f1ad2\" (UID: \"b00beb26-7214-4b13-9734-b440d33f1ad2\") " Jul 22 03:43:42.043064 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.043051 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b00beb26-7214-4b13-9734-b440d33f1ad2-openshift-service-ca\") pod \"b00beb26-7214-4b13-9734-b440d33f1ad2\" (UID: \"b00beb26-7214-4b13-9734-b440d33f1ad2\") " Jul 22 03:43:42.043396 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.043373 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b00beb26-7214-4b13-9734-b440d33f1ad2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b00beb26-7214-4b13-9734-b440d33f1ad2" (UID: "b00beb26-7214-4b13-9734-b440d33f1ad2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:43:42.045053 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.045033 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b00beb26-7214-4b13-9734-b440d33f1ad2-kube-api-access-2hkxz" (OuterVolumeSpecName: "kube-api-access-2hkxz") pod "b00beb26-7214-4b13-9734-b440d33f1ad2" (UID: "b00beb26-7214-4b13-9734-b440d33f1ad2"). InnerVolumeSpecName "kube-api-access-2hkxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:43:42.045163 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.045147 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b00beb26-7214-4b13-9734-b440d33f1ad2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b00beb26-7214-4b13-9734-b440d33f1ad2" (UID: "b00beb26-7214-4b13-9734-b440d33f1ad2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:43:42.144476 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.144448 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b00beb26-7214-4b13-9734-b440d33f1ad2-maas-api-tls\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:43:42.144476 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.144472 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hkxz\" (UniqueName: \"kubernetes.io/projected/b00beb26-7214-4b13-9734-b440d33f1ad2-kube-api-access-2hkxz\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:43:42.144628 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.144483 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b00beb26-7214-4b13-9734-b440d33f1ad2-openshift-service-ca\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:43:42.272548 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.272518 2580 generic.go:356] "Generic (PLEG): container finished" podID="b00beb26-7214-4b13-9734-b440d33f1ad2" containerID="b5af0f8f55526f3878a3ada2104ed303d34ef2d0ddf38bcfb4253829b2744178" exitCode=0 Jul 22 03:43:42.272687 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.272563 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" Jul 22 03:43:42.272687 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.272602 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" event={"ID":"b00beb26-7214-4b13-9734-b440d33f1ad2","Type":"ContainerDied","Data":"b5af0f8f55526f3878a3ada2104ed303d34ef2d0ddf38bcfb4253829b2744178"} Jul 22 03:43:42.272687 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.272676 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh" event={"ID":"b00beb26-7214-4b13-9734-b440d33f1ad2","Type":"ContainerDied","Data":"99bdc20f770c8f2cac68622efe621389ef6d70a89a2eb9335ac4978b3d294d2e"} Jul 22 03:43:42.272811 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.272700 2580 scope.go:117] "RemoveContainer" containerID="b5af0f8f55526f3878a3ada2104ed303d34ef2d0ddf38bcfb4253829b2744178" Jul 22 03:43:42.281112 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.281097 2580 scope.go:117] "RemoveContainer" containerID="b5af0f8f55526f3878a3ada2104ed303d34ef2d0ddf38bcfb4253829b2744178" Jul 22 03:43:42.281341 ip-10-0-138-55 kubenswrapper[2580]: E0722 03:43:42.281322 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5af0f8f55526f3878a3ada2104ed303d34ef2d0ddf38bcfb4253829b2744178\": container with ID starting with b5af0f8f55526f3878a3ada2104ed303d34ef2d0ddf38bcfb4253829b2744178 not found: ID does not exist" containerID="b5af0f8f55526f3878a3ada2104ed303d34ef2d0ddf38bcfb4253829b2744178" Jul 22 03:43:42.281405 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.281347 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5af0f8f55526f3878a3ada2104ed303d34ef2d0ddf38bcfb4253829b2744178"} err="failed to get container status \"b5af0f8f55526f3878a3ada2104ed303d34ef2d0ddf38bcfb4253829b2744178\": rpc error: code = NotFound desc = could not find container \"b5af0f8f55526f3878a3ada2104ed303d34ef2d0ddf38bcfb4253829b2744178\": container with ID starting with b5af0f8f55526f3878a3ada2104ed303d34ef2d0ddf38bcfb4253829b2744178 not found: ID does not exist" Jul 22 03:43:42.294796 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.294744 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh"] Jul 22 03:43:42.301454 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:42.301436 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-c19e4b-646f67cd8-k8nnh"] Jul 22 03:43:43.366638 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:43:43.366601 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b00beb26-7214-4b13-9734-b440d33f1ad2" path="/var/lib/kubelet/pods/b00beb26-7214-4b13-9734-b440d33f1ad2/volumes" Jul 22 03:44:04.227356 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:04.227322 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc"] Jul 22 03:44:04.227971 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:04.227624 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" podUID="f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3" containerName="maas-api" containerID="cri-o://3c38cfc875164525d16b1a0c26236f5f43fbef6bf9d30c047f575877c7be9306" gracePeriod=30 Jul 22 03:44:04.377317 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:04.377286 2580 generic.go:356] "Generic (PLEG): container finished" podID="f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3" containerID="3c38cfc875164525d16b1a0c26236f5f43fbef6bf9d30c047f575877c7be9306" exitCode=0 Jul 22 03:44:04.377507 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:04.377364 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" event={"ID":"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3","Type":"ContainerDied","Data":"3c38cfc875164525d16b1a0c26236f5f43fbef6bf9d30c047f575877c7be9306"} Jul 22 03:44:04.475143 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:04.475121 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" Jul 22 03:44:04.536814 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:04.536728 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjhk6\" (UniqueName: \"kubernetes.io/projected/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-kube-api-access-rjhk6\") pod \"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3\" (UID: \"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3\") " Jul 22 03:44:04.536814 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:04.536783 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-maas-api-tls\") pod \"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3\" (UID: \"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3\") " Jul 22 03:44:04.537069 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:04.536874 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-openshift-service-ca\") pod \"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3\" (UID: \"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3\") " Jul 22 03:44:04.537208 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:04.537182 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3" (UID: "f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:44:04.538777 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:04.538745 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3" (UID: "f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:44:04.538897 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:04.538809 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-kube-api-access-rjhk6" (OuterVolumeSpecName: "kube-api-access-rjhk6") pod "f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3" (UID: "f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3"). InnerVolumeSpecName "kube-api-access-rjhk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:44:04.637551 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:04.637514 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjhk6\" (UniqueName: \"kubernetes.io/projected/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-kube-api-access-rjhk6\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:44:04.637551 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:04.637545 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-maas-api-tls\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:44:04.637551 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:04.637559 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3-openshift-service-ca\") on node \"ip-10-0-138-55.ec2.internal\" DevicePath \"\"" Jul 22 03:44:05.383085 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:05.383053 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" Jul 22 03:44:05.383438 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:05.383093 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc" event={"ID":"f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3","Type":"ContainerDied","Data":"be5cc48fb779444cc064522e8ca28e2cd254c21b68e9d749aff0b326168469b4"} Jul 22 03:44:05.383438 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:05.383145 2580 scope.go:117] "RemoveContainer" containerID="3c38cfc875164525d16b1a0c26236f5f43fbef6bf9d30c047f575877c7be9306" Jul 22 03:44:05.400409 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:05.400381 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc"] Jul 22 03:44:05.404182 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:05.404161 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-49462a-57fd8fb979-mxhxc"] Jul 22 03:44:07.367936 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:07.367902 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3" path="/var/lib/kubelet/pods/f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3/volumes" Jul 22 03:44:23.321201 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:23.321127 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94fc48448-chs9z_f0fe9a9a-4f8a-4b1a-bf18-70417a38239c/copy-manifests/0.log" Jul 22 03:44:23.329122 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:23.329096 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94fc48448-chs9z_f0fe9a9a-4f8a-4b1a-bf18-70417a38239c/manager/0.log" Jul 22 03:44:23.988204 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:23.988174 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6f7dbbff56-d47l8_c3b885e7-35a8-4fe6-958b-8562b91dd8e7/maas-api/0.log" Jul 22 03:44:24.361712 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:24.361685 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-xspnd_2164a2af-74ed-465f-9172-0d8685e7d35f/postgres/0.log" Jul 22 03:44:26.410613 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:26.410578 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-r99n7_e8c6f392-6a4f-4774-8908-6c54cd7130cd/manager/0.log" Jul 22 03:44:27.187799 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:27.187761 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-wm2q8_02f86537-3d6d-4b62-a70f-f44d3e6f3537/discovery/0.log" Jul 22 03:44:28.660199 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:28.660169 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fnfrsr_a1868f38-01ed-4cb6-a787-69cfa5088f80/main/0.log" Jul 22 03:44:28.787552 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:28.787518 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-rpz4l_4d30d12f-3ee6-4b3d-9a5d-28efed8b5abf/main/0.log" Jul 22 03:44:37.861529 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:37.861493 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9tw9c_183fb194-f313-4528-bf1b-c457f3174d9d/global-pull-secret-syncer/0.log" Jul 22 03:44:38.080786 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:38.080752 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qk5x9_0625b87a-9139-4d7e-9a02-78c535b9473a/konnectivity-agent/0.log" Jul 22 03:44:38.198218 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:38.198146 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-55.ec2.internal_671074e8de9b7c15ed7884f108ed8b26/haproxy/0.log" Jul 22 03:44:43.692791 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:43.692691 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-r99n7_e8c6f392-6a4f-4774-8908-6c54cd7130cd/manager/0.log" Jul 22 03:44:45.437542 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:45.437459 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b/alertmanager/0.log" Jul 22 03:44:45.465995 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:45.465963 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b/config-reloader/0.log" Jul 22 03:44:45.490818 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:45.490790 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b/kube-rbac-proxy-web/0.log" Jul 22 03:44:45.528264 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:45.528234 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b/kube-rbac-proxy/0.log" Jul 22 03:44:45.549995 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:45.549951 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b/kube-rbac-proxy-metric/0.log" Jul 22 03:44:45.574823 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:45.574796 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b/prom-label-proxy/0.log" Jul 22 03:44:45.600814 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:45.600778 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3e0f600b-5a7f-4c72-addc-e7f83f9b3a1b/init-config-reloader/0.log" Jul 22 03:44:45.667756 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:45.667723 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-tst5h_3c7ed38f-86c4-4046-8f3e-516265372dcb/cluster-monitoring-operator/0.log" Jul 22 03:44:45.707464 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:45.707378 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-4cf24_3158f020-696e-4d3f-992c-054efbaa67dc/kube-state-metrics/0.log" Jul 22 03:44:45.728404 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:45.728363 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-4cf24_3158f020-696e-4d3f-992c-054efbaa67dc/kube-rbac-proxy-main/0.log" Jul 22 03:44:45.751467 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:45.751438 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-4cf24_3158f020-696e-4d3f-992c-054efbaa67dc/kube-rbac-proxy-self/0.log" Jul 22 03:44:45.853524 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:45.853499 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f96v5_9668d2df-a646-45c2-b675-6b9a0b4586d5/node-exporter/0.log" Jul 22 03:44:45.876034 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:45.876006 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f96v5_9668d2df-a646-45c2-b675-6b9a0b4586d5/kube-rbac-proxy/0.log" Jul 22 03:44:45.900034 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:45.900012 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-f96v5_9668d2df-a646-45c2-b675-6b9a0b4586d5/init-textfile/0.log" Jul 22 03:44:46.190915 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:46.190889 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_813c34cd-40f0-4be0-a8f1-b943263a093b/prometheus/0.log" Jul 22 03:44:46.222568 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:46.222543 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_813c34cd-40f0-4be0-a8f1-b943263a093b/config-reloader/0.log" Jul 22 03:44:46.249049 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:46.249024 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_813c34cd-40f0-4be0-a8f1-b943263a093b/thanos-sidecar/0.log" Jul 22 03:44:46.273348 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:46.273322 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_813c34cd-40f0-4be0-a8f1-b943263a093b/kube-rbac-proxy-web/0.log" Jul 22 03:44:46.295753 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:46.295727 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_813c34cd-40f0-4be0-a8f1-b943263a093b/kube-rbac-proxy/0.log" Jul 22 03:44:46.318160 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:46.318134 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_813c34cd-40f0-4be0-a8f1-b943263a093b/kube-rbac-proxy-thanos/0.log" Jul 22 03:44:46.341365 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:46.341334 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_813c34cd-40f0-4be0-a8f1-b943263a093b/init-config-reloader/0.log" Jul 22 03:44:46.435097 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:46.435069 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-ctjvc_d4693632-3bd9-4778-b305-9ce70fe4b162/prometheus-operator-admission-webhook/0.log" Jul 22 03:44:48.138108 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.138072 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl"] Jul 22 03:44:48.138489 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.138415 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3" containerName="maas-api" Jul 22 03:44:48.138489 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.138426 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3" containerName="maas-api" Jul 22 03:44:48.138489 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.138441 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b00beb26-7214-4b13-9734-b440d33f1ad2" containerName="maas-api" Jul 22 03:44:48.138489 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.138446 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b00beb26-7214-4b13-9734-b440d33f1ad2" containerName="maas-api" Jul 22 03:44:48.138616 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.138510 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b00beb26-7214-4b13-9734-b440d33f1ad2" containerName="maas-api" Jul 22 03:44:48.138616 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.138519 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f83ab8eb-eab5-4f61-8cc5-1c0b5ebf5de3" containerName="maas-api" Jul 22 03:44:48.142123 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.142104 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:48.144376 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.144351 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kwn6d\"/\"default-dockercfg-qqsd7\"" Jul 22 03:44:48.144521 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.144351 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kwn6d\"/\"kube-root-ca.crt\"" Jul 22 03:44:48.145139 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.145123 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kwn6d\"/\"openshift-service-ca.crt\"" Jul 22 03:44:48.152641 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.152622 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl"] Jul 22 03:44:48.319063 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.319031 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/776c13ed-2a02-4722-9f4d-bb7e3f428550-lib-modules\") pod \"perf-node-gather-daemonset-zbzpl\" (UID: \"776c13ed-2a02-4722-9f4d-bb7e3f428550\") " pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:48.319243 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.319099 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/776c13ed-2a02-4722-9f4d-bb7e3f428550-sys\") pod \"perf-node-gather-daemonset-zbzpl\" (UID: \"776c13ed-2a02-4722-9f4d-bb7e3f428550\") " pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:48.319243 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.319133 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/776c13ed-2a02-4722-9f4d-bb7e3f428550-podres\") pod \"perf-node-gather-daemonset-zbzpl\" (UID: \"776c13ed-2a02-4722-9f4d-bb7e3f428550\") " pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:48.319243 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.319160 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n954g\" (UniqueName: \"kubernetes.io/projected/776c13ed-2a02-4722-9f4d-bb7e3f428550-kube-api-access-n954g\") pod \"perf-node-gather-daemonset-zbzpl\" (UID: \"776c13ed-2a02-4722-9f4d-bb7e3f428550\") " pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:48.319355 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.319241 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/776c13ed-2a02-4722-9f4d-bb7e3f428550-proc\") pod \"perf-node-gather-daemonset-zbzpl\" (UID: \"776c13ed-2a02-4722-9f4d-bb7e3f428550\") " pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:48.420121 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.420033 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/776c13ed-2a02-4722-9f4d-bb7e3f428550-sys\") pod \"perf-node-gather-daemonset-zbzpl\" (UID: \"776c13ed-2a02-4722-9f4d-bb7e3f428550\") " pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:48.420278 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.420134 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/776c13ed-2a02-4722-9f4d-bb7e3f428550-sys\") pod \"perf-node-gather-daemonset-zbzpl\" (UID: \"776c13ed-2a02-4722-9f4d-bb7e3f428550\") " pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:48.420278 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.420140 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/776c13ed-2a02-4722-9f4d-bb7e3f428550-podres\") pod \"perf-node-gather-daemonset-zbzpl\" (UID: \"776c13ed-2a02-4722-9f4d-bb7e3f428550\") " pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:48.420278 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.420226 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n954g\" (UniqueName: \"kubernetes.io/projected/776c13ed-2a02-4722-9f4d-bb7e3f428550-kube-api-access-n954g\") pod \"perf-node-gather-daemonset-zbzpl\" (UID: \"776c13ed-2a02-4722-9f4d-bb7e3f428550\") " pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:48.420278 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.420255 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/776c13ed-2a02-4722-9f4d-bb7e3f428550-podres\") pod \"perf-node-gather-daemonset-zbzpl\" (UID: \"776c13ed-2a02-4722-9f4d-bb7e3f428550\") " pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:48.420278 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.420266 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/776c13ed-2a02-4722-9f4d-bb7e3f428550-proc\") pod \"perf-node-gather-daemonset-zbzpl\" (UID: \"776c13ed-2a02-4722-9f4d-bb7e3f428550\") " pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:48.420503 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.420347 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/776c13ed-2a02-4722-9f4d-bb7e3f428550-lib-modules\") pod \"perf-node-gather-daemonset-zbzpl\" (UID: \"776c13ed-2a02-4722-9f4d-bb7e3f428550\") " pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:48.420503 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.420383 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/776c13ed-2a02-4722-9f4d-bb7e3f428550-proc\") pod \"perf-node-gather-daemonset-zbzpl\" (UID: \"776c13ed-2a02-4722-9f4d-bb7e3f428550\") " pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:48.420608 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.420511 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/776c13ed-2a02-4722-9f4d-bb7e3f428550-lib-modules\") pod \"perf-node-gather-daemonset-zbzpl\" (UID: \"776c13ed-2a02-4722-9f4d-bb7e3f428550\") " pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:48.429906 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.429879 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n954g\" (UniqueName: \"kubernetes.io/projected/776c13ed-2a02-4722-9f4d-bb7e3f428550-kube-api-access-n954g\") pod \"perf-node-gather-daemonset-zbzpl\" (UID: \"776c13ed-2a02-4722-9f4d-bb7e3f428550\") " pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:48.452851 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.452817 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:48.553779 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.553722 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/1.log" Jul 22 03:44:48.563770 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.563748 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-94dxd_fdb88bbb-3960-48f5-b481-9a8c5a0c5fa6/console-operator/2.log" Jul 22 03:44:48.584039 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.583946 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl"] Jul 22 03:44:48.586653 ip-10-0-138-55 kubenswrapper[2580]: W0722 03:44:48.586624 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod776c13ed_2a02_4722_9f4d_bb7e3f428550.slice/crio-744927d4272576634f37db6e42c5a088ae4cfb0925cf1986e2da90e413889dac WatchSource:0}: Error finding container 744927d4272576634f37db6e42c5a088ae4cfb0925cf1986e2da90e413889dac: Status 404 returned error can't find the container with id 744927d4272576634f37db6e42c5a088ae4cfb0925cf1986e2da90e413889dac Jul 22 03:44:48.592444 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:48.592412 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" event={"ID":"776c13ed-2a02-4722-9f4d-bb7e3f428550","Type":"ContainerStarted","Data":"744927d4272576634f37db6e42c5a088ae4cfb0925cf1986e2da90e413889dac"} Jul 22 03:44:49.097695 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:49.097615 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5b8848d99f-52bps_a7b1a3e5-9e1b-4c82-a56a-ff396c2ec606/console/0.log" Jul 22 03:44:49.140205 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:49.140153 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-vd8sx_8c542afe-969b-42c5-9dc5-dc4eb2586201/download-server/0.log" Jul 22 03:44:49.598230 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:49.598193 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" event={"ID":"776c13ed-2a02-4722-9f4d-bb7e3f428550","Type":"ContainerStarted","Data":"5268370b5a6d9c2729a00bcb0518cc4951e4e0b567466bde4991b09713af0e01"} Jul 22 03:44:49.598410 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:49.598333 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:49.616698 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:49.616652 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" podStartSLOduration=1.616637509 podStartE2EDuration="1.616637509s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:44:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:44:49.614362442 +0000 UTC m=+2818.805556746" watchObservedRunningTime="2026-07-22 03:44:49.616637509 +0000 UTC m=+2818.807831846" Jul 22 03:44:49.728901 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:49.728874 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-frhxj_74a3c85e-05c0-4ad6-addb-c70c0996ea10/volume-data-source-validator/0.log" Jul 22 03:44:50.897629 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:50.897593 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b46cz_33beac89-d63c-4c71-82db-1790e90a1447/dns/0.log" Jul 22 03:44:50.921684 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:50.921658 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b46cz_33beac89-d63c-4c71-82db-1790e90a1447/kube-rbac-proxy/0.log" Jul 22 03:44:50.974279 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:50.974253 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7t98c_308ab8f1-8fb2-4833-b583-b22cf8e3cddd/dns-node-resolver/0.log" Jul 22 03:44:51.546472 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:51.546446 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-99vc7_91a9bdc2-87c0-47f1-93d7-1f6ba5ba524b/node-ca/0.log" Jul 22 03:44:52.809205 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:52.809171 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-wm2q8_02f86537-3d6d-4b62-a70f-f44d3e6f3537/discovery/0.log" Jul 22 03:44:53.710581 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:53.710551 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r6wl8_7981d361-b9b8-4e56-97ad-928079266fb8/serve-healthcheck-canary/0.log" Jul 22 03:44:54.346585 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:54.346550 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pgnvv_7018409a-3816-4591-a676-eb0e71ffed9d/kube-rbac-proxy/0.log" Jul 22 03:44:54.366743 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:54.366719 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pgnvv_7018409a-3816-4591-a676-eb0e71ffed9d/exporter/0.log" Jul 22 03:44:54.392290 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:54.392264 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pgnvv_7018409a-3816-4591-a676-eb0e71ffed9d/extractor/0.log" Jul 22 03:44:55.612461 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:55.612435 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kwn6d/perf-node-gather-daemonset-zbzpl" Jul 22 03:44:57.227418 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:57.227388 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94fc48448-chs9z_f0fe9a9a-4f8a-4b1a-bf18-70417a38239c/manager/0.log" Jul 22 03:44:57.251553 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:57.251496 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-94fc48448-chs9z_f0fe9a9a-4f8a-4b1a-bf18-70417a38239c/copy-manifests/0.log" Jul 22 03:44:58.820756 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:58.820690 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-58c8df6b5b-vjsdz_36df533d-c856-4943-8b75-bb07342c17c0/manager/0.log" Jul 22 03:44:58.848822 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:58.848795 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-jkrs7_b2c39a89-f0f9-43c4-af36-26578cc8e90d/openshift-lws-operator/0.log" Jul 22 03:44:58.849176 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:44:58.849154 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-wm2q8_02f86537-3d6d-4b62-a70f-f44d3e6f3537/discovery/0.log" Jul 22 03:45:00.477315 ip-10-0-138-55 kubenswrapper[2580]: I0722 03:45:00.477286 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-wm2q8_02f86537-3d6d-4b62-a70f-f44d3e6f3537/discovery/0.log"