Jul 18 01:46:01.609611 ip-10-0-132-151 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 18 01:46:01.609622 ip-10-0-132-151 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 18 01:46:01.609639 ip-10-0-132-151 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 18 01:46:01.610060 ip-10-0-132-151 systemd[1]: Failed to start Kubernetes Kubelet. Jul 18 01:46:10.800806 ip-10-0-132-151 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jul 18 01:46:10.800819 ip-10-0-132-151 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 5ec512f3352b400dba29135dd6963ccd -- Jul 18 01:47:38.678230 ip-10-0-132-151 systemd[1]: Starting Kubernetes Kubelet... Jul 18 01:47:39.171542 ip-10-0-132-151 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 01:47:39.171542 ip-10-0-132-151 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 18 01:47:39.171542 ip-10-0-132-151 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 01:47:39.171542 ip-10-0-132-151 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 18 01:47:39.171542 ip-10-0-132-151 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 01:47:39.171542 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.155367 2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 18 01:47:39.171542 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.162951 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:47:39.171542 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.162967 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:47:39.171542 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.162972 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:47:39.171542 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.162976 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:47:39.171542 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.162981 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:47:39.171542 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.162987 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:47:39.171542 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.162991 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.162995 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.162998 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163002 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163007 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163011 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163015 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163026 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163033 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163039 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163044 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163048 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163052 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163057 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163061 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163065 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163069 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163075 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163080 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:47:39.180233 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163083 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163089 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163094 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163098 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163102 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163105 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163109 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163113 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163117 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163121 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163125 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163128 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163132 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163136 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163140 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163143 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163147 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163152 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163156 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163160 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:47:39.180824 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163164 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163168 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163172 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163176 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163179 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163183 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163187 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163191 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163195 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163198 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163202 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163205 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163209 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163213 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163217 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163222 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163226 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163230 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163233 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163237 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:47:39.182338 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163241 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163245 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163249 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163252 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163255 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163259 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163263 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163267 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163271 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163275 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163279 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163283 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163286 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163290 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163294 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163298 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163302 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163306 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163309 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163312 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:47:39.184354 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163316 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163320 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163324 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163328 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163331 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163336 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163339 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163343 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163349 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163354 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163359 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163363 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163366 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.163370 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164349 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164358 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164363 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164367 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164371 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:47:39.186483 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164375 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164379 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164383 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164387 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164393 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164397 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164401 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164404 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164409 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164412 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164416 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164420 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164423 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164429 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164434 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164439 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164444 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164448 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164451 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:47:39.193695 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164455 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164459 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164463 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164467 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164471 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164483 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164487 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164491 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164497 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164501 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164509 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164513 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164519 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164524 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164528 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164532 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164536 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164539 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164545 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:47:39.194281 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164549 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164553 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164556 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164560 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164563 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164568 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164572 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164576 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164580 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164583 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164587 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164590 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164594 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164598 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164601 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164605 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164609 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164612 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164616 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164620 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:47:39.194742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164623 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164627 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164630 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164636 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164640 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164643 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164647 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164651 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164654 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164658 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164662 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164666 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164670 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164674 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164678 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164682 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164685 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164688 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164692 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164696 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:47:39.195247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164700 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164703 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164707 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164710 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164714 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164717 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164721 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164725 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164729 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164732 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164736 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164757 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164761 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164764 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164768 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.164773 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165295 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165302 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165306 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165309 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:47:39.195743 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165312 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165315 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165317 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165322 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165325 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165328 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165331 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165334 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165336 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165339 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165341 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165344 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165346 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165349 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165352 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165354 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165357 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165360 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165362 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165365 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:47:39.196238 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165367 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165369 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165372 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165374 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165377 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165379 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165382 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165386 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165388 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165391 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165393 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165396 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165399 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165401 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165404 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165407 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165410 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165415 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165420 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165423 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:47:39.196720 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165425 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165428 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165430 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165433 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165435 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165438 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165441 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165443 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165446 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165449 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165451 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165454 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165456 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165459 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165461 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165464 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165466 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165469 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165471 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165476 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:47:39.197214 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165478 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165482 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165485 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165488 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165491 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165493 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165496 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165499 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165502 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165504 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165507 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165510 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165512 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165515 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165517 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165520 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165522 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165525 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165527 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:47:39.197715 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165531 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165534 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165537 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165540 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165557 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165561 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165564 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165567 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165569 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165572 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165575 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165577 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165582 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165584 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165587 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.165589 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165656 2570 flags.go:64] FLAG: --address="0.0.0.0" Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165662 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165668 2570 flags.go:64] FLAG: --anonymous-auth="true" Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165673 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165679 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 18 01:47:39.198190 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165682 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165686 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165691 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165695 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165698 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165702 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165705 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165708 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165711 2570 flags.go:64] FLAG: --cgroup-root="" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165714 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165718 2570 flags.go:64] FLAG: --client-ca-file="" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165721 2570 flags.go:64] FLAG: --cloud-provider="external" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165724 2570 flags.go:64] FLAG: --cluster-dns="[]" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165729 2570 flags.go:64] FLAG: --cluster-domain="" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165731 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165735 2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165738 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165742 2570 flags.go:64] FLAG: --container-log-max-files="5" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165746 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165749 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165752 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165755 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165758 2570 flags.go:64] FLAG: --contention-profiling="false" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165762 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 18 01:47:39.198807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165765 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165769 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165772 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165776 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165779 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165782 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165785 2570 flags.go:64] FLAG: --enable-load-reader="false" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165790 2570 flags.go:64] FLAG: --enable-server="true" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165793 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165798 2570 flags.go:64] FLAG: --event-burst="100" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165801 2570 flags.go:64] FLAG: --event-qps="50" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165804 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165808 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165811 2570 flags.go:64] FLAG: --eviction-hard="" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165815 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165818 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165821 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165824 2570 flags.go:64] FLAG: --eviction-soft="" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165827 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165830 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165833 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165836 2570 flags.go:64] FLAG: --experimental-mounter-path="" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165839 2570 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165842 2570 flags.go:64] FLAG: --fail-swap-on="true" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165845 2570 flags.go:64] FLAG: --feature-gates="" Jul 18 01:47:39.199489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165848 2570 flags.go:64] FLAG: --file-check-frequency="20s" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165851 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165855 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165858 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165861 2570 flags.go:64] FLAG: --healthz-port="10248" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165864 2570 flags.go:64] FLAG: --help="false" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165868 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-132-151.ec2.internal" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165871 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165874 2570 flags.go:64] FLAG: --http-check-frequency="20s" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165877 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165880 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165884 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165887 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165889 2570 flags.go:64] FLAG: --image-service-endpoint="" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165894 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165896 2570 flags.go:64] FLAG: --kube-api-burst="100" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165899 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165903 2570 flags.go:64] FLAG: --kube-api-qps="50" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165906 2570 flags.go:64] FLAG: --kube-reserved="" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165909 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165912 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165935 2570 flags.go:64] FLAG: --kubelet-cgroups="" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165940 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165945 2570 flags.go:64] FLAG: --lock-file="" Jul 18 01:47:39.200197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165949 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165953 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165956 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165962 2570 flags.go:64] FLAG: --log-json-split-stream="false" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165965 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165968 2570 flags.go:64] FLAG: --log-text-split-stream="false" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165970 2570 flags.go:64] FLAG: --logging-format="text" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165973 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165977 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165980 2570 flags.go:64] FLAG: --manifest-url="" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165983 2570 flags.go:64] FLAG: --manifest-url-header="" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165987 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165990 2570 flags.go:64] FLAG: --max-open-files="1000000" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165994 2570 flags.go:64] FLAG: --max-pods="110" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.165999 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166002 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166005 2570 flags.go:64] FLAG: --memory-manager-policy="None" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166008 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166011 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166014 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166017 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166025 2570 flags.go:64] FLAG: --node-status-max-images="50" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166029 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166032 2570 flags.go:64] FLAG: --oom-score-adj="-999" Jul 18 01:47:39.200814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166035 2570 flags.go:64] FLAG: --pod-cidr="" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166038 2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166044 2570 flags.go:64] FLAG: --pod-manifest-path="" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166047 2570 flags.go:64] FLAG: --pod-max-pids="-1" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166050 2570 flags.go:64] FLAG: --pods-per-core="0" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166054 2570 flags.go:64] FLAG: --port="10250" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166057 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166060 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ad230c7b6f5e1854" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166063 2570 flags.go:64] FLAG: --qos-reserved="" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166066 2570 flags.go:64] FLAG: --read-only-port="10255" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166069 2570 flags.go:64] FLAG: --register-node="true" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166072 2570 flags.go:64] FLAG: --register-with-taints="" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166076 2570 flags.go:64] FLAG: --registry-burst="10" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166079 2570 flags.go:64] FLAG: --registry-qps="5" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166082 2570 flags.go:64] FLAG: --reserved-cpus="" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166084 2570 flags.go:64] FLAG: --reserved-memory="" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166088 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166091 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166094 2570 flags.go:64] FLAG: --rotate-certificates="false" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166097 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166100 2570 flags.go:64] FLAG: --runonce="false" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166103 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166107 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166110 2570 flags.go:64] FLAG: --seccomp-default="false" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166114 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 18 01:47:39.201735 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166117 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166120 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166123 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166126 2570 flags.go:64] FLAG: --storage-driver-password="root" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166129 2570 flags.go:64] FLAG: --storage-driver-secure="false" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166132 2570 flags.go:64] FLAG: --storage-driver-table="stats" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166137 2570 flags.go:64] FLAG: --storage-driver-user="root" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166140 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166143 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166147 2570 flags.go:64] FLAG: --system-cgroups="" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166149 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166155 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166158 2570 flags.go:64] FLAG: --tls-cert-file="" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166161 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166166 2570 flags.go:64] FLAG: --tls-min-version="" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166169 2570 flags.go:64] FLAG: --tls-private-key-file="" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166172 2570 flags.go:64] FLAG: --topology-manager-policy="none" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166175 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166177 2570 flags.go:64] FLAG: --topology-manager-scope="container" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166180 2570 flags.go:64] FLAG: --v="2" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166184 2570 flags.go:64] FLAG: --version="false" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166188 2570 flags.go:64] FLAG: --vmodule="" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166192 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.166195 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166309 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:47:39.202742 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166312 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166316 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166319 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166322 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166327 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166330 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166332 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166335 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166338 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166341 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166344 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166347 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166350 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166354 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166357 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166360 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166362 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166365 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166368 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166371 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:47:39.203809 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166373 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166376 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166379 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166381 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166384 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166386 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166389 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166391 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166394 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166397 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166399 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166401 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166406 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166409 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166412 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166415 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166419 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166421 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166423 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:47:39.204567 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166426 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166429 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166431 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166434 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166436 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166439 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166443 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166446 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166448 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166451 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166453 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166456 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166459 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166461 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166464 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166467 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166470 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166472 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166475 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166477 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:47:39.205132 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166480 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166482 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166485 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166487 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166490 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166492 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166494 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166497 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166499 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166504 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166506 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166508 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166512 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166516 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166519 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166521 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166524 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166526 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166530 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166533 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:47:39.205819 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166536 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166539 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166541 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166544 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166546 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166549 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166551 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166554 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166557 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166560 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166562 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166565 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166568 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166572 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166575 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166578 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166580 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166583 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:47:39.206389 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.166586 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:47:39.206877 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.167352 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 01:47:39.206877 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.174808 2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 18 01:47:39.206877 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.174825 2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 18 01:47:39.206877 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.174844 2570 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 18 01:47:39.206877 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.174849 2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 18 01:47:39.206877 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174926 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:47:39.206877 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174933 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:47:39.206877 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174936 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:47:39.206877 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174939 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:47:39.206877 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174942 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:47:39.206877 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174945 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:47:39.206877 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174948 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:47:39.206877 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174951 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:47:39.206877 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174954 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:47:39.206877 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174957 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174960 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174963 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174965 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174968 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174971 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174973 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174983 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174986 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174988 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174991 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174994 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174996 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.174998 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175001 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175003 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175007 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175010 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175013 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175016 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:47:39.207400 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175019 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175022 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175025 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175027 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175030 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175032 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175035 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175037 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175040 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175042 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175045 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175048 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175050 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175056 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175059 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175061 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175064 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175066 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175069 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175071 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:47:39.207893 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175079 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175082 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175085 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175089 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175095 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175098 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175101 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175104 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175107 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175109 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175112 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175114 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175117 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175120 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175122 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175126 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175130 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175133 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175137 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:47:39.208383 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175141 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175144 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175147 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175150 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175153 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175156 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175159 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175161 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175164 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175166 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175169 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175171 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175174 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175176 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175184 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175187 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175190 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175194 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175197 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175212 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:47:39.208832 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175215 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:47:39.209331 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175218 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:47:39.209331 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175221 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:47:39.209331 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175224 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:47:39.209331 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175226 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:47:39.209331 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175229 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:47:39.209331 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175232 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:47:39.209331 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175234 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:47:39.209331 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175237 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:47:39.209331 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175239 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:47:39.209331 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175242 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:47:39.209331 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.175247 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 01:47:39.209331 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175396 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:47:39.209331 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175401 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:47:39.209331 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175404 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:47:39.209331 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175407 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175410 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175412 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175415 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175418 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175420 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175423 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175426 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175428 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175431 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175434 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175436 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175448 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175453 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175459 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175462 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175464 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175468 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175470 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:47:39.209699 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175473 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175476 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175479 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175481 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175484 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175486 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175489 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175492 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175494 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175497 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175499 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175502 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175505 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175507 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175510 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175520 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175522 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175525 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175528 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175530 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:47:39.210180 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175533 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175536 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175538 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175541 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175543 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175546 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175555 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175560 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175562 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175565 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175567 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175570 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175574 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175577 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175580 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175582 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175585 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175589 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175592 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:47:39.210667 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175595 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175598 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175601 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175604 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175607 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175610 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175613 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175615 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175618 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175620 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175622 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175625 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175627 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175630 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175633 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175635 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175638 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175640 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175643 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175645 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:47:39.211143 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175655 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:47:39.211624 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175657 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:47:39.211624 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175660 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:47:39.211624 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175663 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:47:39.211624 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175665 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:47:39.211624 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175669 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:47:39.211624 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175672 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:47:39.211624 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175675 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:47:39.211624 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175677 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:47:39.211624 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175680 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:47:39.211624 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175682 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:47:39.211624 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175685 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:47:39.211624 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175688 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:47:39.211624 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175690 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:47:39.211624 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175693 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:47:39.211624 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175695 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:47:39.211624 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175698 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:47:39.211624 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:39.175700 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:47:39.212077 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.175705 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 01:47:39.212077 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.175828 2570 server.go:962] "Client rotation is on, will bootstrap in background" Jul 18 01:47:39.212077 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.178278 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 18 01:47:39.212077 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.179958 2570 server.go:1019] "Starting client certificate rotation" Jul 18 01:47:39.212077 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.180283 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 01:47:39.212077 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.181048 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 01:47:39.212077 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.186510 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 01:47:39.212077 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.191331 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 01:47:39.212077 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.205156 2570 log.go:25] "Validated CRI v1 runtime API" Jul 18 01:47:39.212077 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.207741 2570 log.go:25] "Validated CRI v1 image API" Jul 18 01:47:39.212077 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.208611 2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 18 01:47:39.212497 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.212470 2570 fs.go:135] Filesystem UUIDs: map[3b704ef7-ebf6-4ef9-932d-90b5c46396a7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c46e4339-1945-4308-a76a-8d26ca9a7363:/dev/nvme0n1p3] Jul 18 01:47:39.212529 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.212498 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 18 01:47:39.215977 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.215960 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 01:47:39.218368 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.218253 2570 manager.go:216] Machine: {Timestamp:2026-07-18 01:47:39.216851779 +0000 UTC m=+0.410999916 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099764 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec208c5b98690eea2c26215a3fe7394c SystemUUID:ec208c5b-9869-0eea-2c26-215a3fe7394c BootID:5ec512f3-352b-400d-ba29-135dd6963ccd Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1a:9f:cf:ce:f5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1a:9f:cf:ce:f5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e2:98:6e:35:f2:7a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 18 01:47:39.218368 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.218366 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 18 01:47:39.218470 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.218448 2570 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 18 01:47:39.221012 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.220989 2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 18 01:47:39.221145 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.221014 2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-151.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 18 01:47:39.221818 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.221809 2570 topology_manager.go:138] "Creating topology manager with none policy" Jul 18 01:47:39.221848 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.221819 2570 container_manager_linux.go:309] "Creating device plugin manager" Jul 18 01:47:39.221848 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.221827 2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 01:47:39.222630 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.222620 2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 01:47:39.224494 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.224484 2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 18 01:47:39.229144 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.229127 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 18 01:47:39.229295 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.229285 2570 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 18 01:47:39.229336 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.229327 2570 kubelet.go:486] "Attempting to sync node with API server" Jul 18 01:47:39.229365 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.229358 2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 18 01:47:39.229402 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.229391 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 18 01:47:39.229433 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.229409 2570 kubelet.go:398] "Adding apiserver pod source" Jul 18 01:47:39.229433 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.229424 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 18 01:47:39.229771 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.229748 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 01:47:39.229771 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.229772 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 01:47:39.233486 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.233469 2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 18 01:47:39.236110 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.236095 2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 18 01:47:39.236193 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.236126 2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 18 01:47:39.237577 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.237565 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 18 01:47:39.237627 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.237581 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 18 01:47:39.237627 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.237588 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 18 01:47:39.237627 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.237594 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 18 01:47:39.237627 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.237605 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 18 01:47:39.237627 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.237614 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 18 01:47:39.237627 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.237623 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 18 01:47:39.237627 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.237628 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 18 01:47:39.237807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.237635 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 18 01:47:39.237807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.237640 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 18 01:47:39.237807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.237648 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 18 01:47:39.237807 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.237654 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 18 01:47:39.238548 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.238538 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 18 01:47:39.238548 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.238548 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 18 01:47:39.241782 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.241767 2570 server.go:1268] "Started kubelet" Jul 18 01:47:39.241950 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.241882 2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 18 01:47:39.242024 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.241973 2570 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 18 01:47:39.247183 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.247142 2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 18 01:47:39.247398 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:39.247359 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 18 01:47:39.247583 ip-10-0-132-151 systemd[1]: Started Kubernetes Kubelet. Jul 18 01:47:39.249490 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:39.247608 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-151.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 18 01:47:39.249490 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.249485 2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-151.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 18 01:47:39.249964 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.249933 2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 18 01:47:39.250516 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:39.250495 2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 18 01:47:39.251094 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.251078 2570 server.go:310] "Adding debug handlers to kubelet server" Jul 18 01:47:39.255846 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:39.254734 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-151.ec2.internal.18c33f1a39e33c9b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-151.ec2.internal,UID:ip-10-0-132-151.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-151.ec2.internal,},FirstTimestamp:2026-07-18 01:47:39.241741467 +0000 UTC m=+0.435889601,LastTimestamp:2026-07-18 01:47:39.241741467 +0000 UTC m=+0.435889601,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-151.ec2.internal,}" Jul 18 01:47:39.259350 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.259302 2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 18 01:47:39.259350 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.259313 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 18 01:47:39.259488 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:39.259388 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 18 01:47:39.260156 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.260133 2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 18 01:47:39.260156 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.260134 2570 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 18 01:47:39.260156 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.260161 2570 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 18 01:47:39.260363 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.260214 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 18 01:47:39.260363 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.260229 2570 factory.go:55] Registering systemd factory Jul 18 01:47:39.260363 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.260237 2570 factory.go:223] Registration of the systemd container factory successfully Jul 18 01:47:39.260363 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.260273 2570 reconstruct.go:97] "Volume reconstruction finished" Jul 18 01:47:39.260363 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.260282 2570 reconciler.go:29] "Reconciler: start to sync state" Jul 18 01:47:39.260542 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.260472 2570 factory.go:153] Registering CRI-O factory Jul 18 01:47:39.260542 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.260485 2570 factory.go:223] Registration of the crio container factory successfully Jul 18 01:47:39.260542 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.260506 2570 factory.go:103] Registering Raw factory Jul 18 01:47:39.260542 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.260519 2570 manager.go:1196] Started watching for new ooms in manager Jul 18 01:47:39.261090 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:39.261069 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 18 01:47:39.261214 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.261201 2570 manager.go:349] Starting recovery of all containers Jul 18 01:47:39.261400 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:39.261379 2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-151.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 18 01:47:39.261506 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.261490 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p4mmd" Jul 18 01:47:39.266846 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.266825 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-p4mmd" Jul 18 01:47:39.273013 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.272985 2570 manager.go:354] Recovery completed Jul 18 01:47:39.277436 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.277422 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:47:39.279274 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.279260 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:47:39.279319 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.279288 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:47:39.279319 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.279299 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:47:39.279810 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.279794 2570 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 18 01:47:39.279810 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.279807 2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 18 01:47:39.279948 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.279828 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 18 01:47:39.279948 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.279831 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:47:39.280522 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.280508 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:47:39.280574 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.280533 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:47:39.280574 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.280552 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:47:39.281975 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.281958 2570 policy_none.go:49] "None policy: Start" Jul 18 01:47:39.282037 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.281979 2570 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 18 01:47:39.282037 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.281990 2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 18 01:47:39.283891 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.283881 2570 policy_none.go:47] "Start" Jul 18 01:47:39.320522 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.320504 2570 manager.go:337] "Starting Device Plugin manager" Jul 18 01:47:39.339576 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:39.320539 2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 18 01:47:39.339576 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.320548 2570 server.go:87] "Starting device plugin registration server" Jul 18 01:47:39.339576 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.320785 2570 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 18 01:47:39.339576 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.320798 2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 18 01:47:39.339576 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.320888 2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 18 01:47:39.339576 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.320994 2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 18 01:47:39.339576 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.321006 2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 18 01:47:39.339576 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:39.321571 2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 18 01:47:39.339576 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:39.321620 2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-151.ec2.internal\" not found" Jul 18 01:47:39.369778 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.369747 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 18 01:47:39.371030 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.371008 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 18 01:47:39.371137 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.371043 2570 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 18 01:47:39.371137 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.371070 2570 kubelet.go:2448] "Starting kubelet main sync loop" Jul 18 01:47:39.371137 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:39.371111 2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 18 01:47:39.373405 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.373385 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:47:39.421224 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.421187 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:47:39.422290 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.422272 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:47:39.422405 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.422303 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:47:39.422405 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.422318 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:47:39.422405 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.422347 2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-151.ec2.internal" Jul 18 01:47:39.433977 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.433961 2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-151.ec2.internal" Jul 18 01:47:39.434024 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:39.433983 2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-151.ec2.internal\": node \"ip-10-0-132-151.ec2.internal\" not found" Jul 18 01:47:39.452871 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:39.452842 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 18 01:47:39.471811 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.471782 2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal"] Jul 18 01:47:39.471945 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.471848 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:47:39.473336 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.473322 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:47:39.473422 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.473350 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:47:39.473422 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.473360 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:47:39.475747 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.475728 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:47:39.475843 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.475828 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 18 01:47:39.475902 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.475856 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:47:39.476496 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.476480 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:47:39.476496 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.476491 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:47:39.476645 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.476506 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:47:39.476645 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.476514 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:47:39.476645 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.476528 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:47:39.476645 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.476516 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:47:39.478930 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.478900 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 18 01:47:39.479022 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.478937 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:47:39.479566 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.479551 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:47:39.479654 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.479580 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:47:39.479654 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.479595 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:47:39.504637 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:39.504609 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-151.ec2.internal\" not found" node="ip-10-0-132-151.ec2.internal" Jul 18 01:47:39.508874 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:39.508854 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-151.ec2.internal\" not found" node="ip-10-0-132-151.ec2.internal" Jul 18 01:47:39.553735 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:39.553657 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 18 01:47:39.561307 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.561286 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15fcf4b6dde762c7ce3ec0cc2a1506cd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"15fcf4b6dde762c7ce3ec0cc2a1506cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 18 01:47:39.561394 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.561314 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/90ea7be93df65d66da652d9417af881e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-151.ec2.internal\" (UID: \"90ea7be93df65d66da652d9417af881e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 18 01:47:39.561394 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:39.561344 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15fcf4b6dde762c7ce3ec0cc2a1506cd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"15fcf4b6dde762c7ce3ec0cc2a1506cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 18 01:47:41.171197 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:41.171160 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 18 01:47:41.178508 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.178486 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15fcf4b6dde762c7ce3ec0cc2a1506cd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"15fcf4b6dde762c7ce3ec0cc2a1506cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 18 01:47:41.178605 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.178521 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/90ea7be93df65d66da652d9417af881e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-151.ec2.internal\" (UID: \"90ea7be93df65d66da652d9417af881e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 18 01:47:41.178605 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.178551 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15fcf4b6dde762c7ce3ec0cc2a1506cd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"15fcf4b6dde762c7ce3ec0cc2a1506cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 18 01:47:41.178605 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.178594 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/15fcf4b6dde762c7ce3ec0cc2a1506cd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"15fcf4b6dde762c7ce3ec0cc2a1506cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 18 01:47:41.178605 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.178601 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/15fcf4b6dde762c7ce3ec0cc2a1506cd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal\" (UID: \"15fcf4b6dde762c7ce3ec0cc2a1506cd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 18 01:47:41.178764 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.178608 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/90ea7be93df65d66da652d9417af881e-config\") pod \"kube-apiserver-proxy-ip-10-0-132-151.ec2.internal\" (UID: \"90ea7be93df65d66da652d9417af881e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 18 01:47:41.271932 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:41.271888 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 18 01:47:41.324432 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.324354 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 18 01:47:41.329429 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.329269 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 18 01:47:41.372698 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:41.372664 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 18 01:47:41.473169 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:41.473135 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 18 01:47:41.573697 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:41.573670 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 18 01:47:41.674302 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:41.674238 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-151.ec2.internal\" not found" Jul 18 01:47:41.678904 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.678879 2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:47:41.697802 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.697767 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 18 01:47:41.698238 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.697911 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 01:47:41.698238 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.697943 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 01:47:41.747122 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.747089 2570 apiserver.go:52] "Watching apiserver" Jul 18 01:47:41.766568 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.766534 2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 18 01:47:41.768130 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.768106 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-8xjzh","openshift-image-registry/node-ca-566ms","openshift-ovn-kubernetes/ovnkube-node-jshl6","kube-system/konnectivity-agent-wj9lj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl","openshift-cluster-node-tuning-operator/tuned-w5hwx","openshift-multus/multus-additional-cni-plugins-5lxc9","openshift-multus/multus-xbkpg","openshift-multus/network-metrics-daemon-5nqdq","openshift-network-diagnostics/network-check-target-2shnl","openshift-network-operator/iptables-alerter-kj7rk","kube-system/global-pull-secret-syncer-bdkrt"] Jul 18 01:47:41.770569 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.770547 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-566ms" Jul 18 01:47:41.772742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.772719 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.773252 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.773217 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 18 01:47:41.773361 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.773284 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 18 01:47:41.775062 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.774010 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 18 01:47:41.775062 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.774244 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tkjhz\"" Jul 18 01:47:41.775209 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.775130 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-f4bzj\"" Jul 18 01:47:41.776013 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.775984 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 18 01:47:41.776118 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.776021 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 18 01:47:41.776174 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.776136 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" Jul 18 01:47:41.776174 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.776154 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 18 01:47:41.776382 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.776365 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8xjzh" Jul 18 01:47:41.778493 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.778473 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 18 01:47:41.778713 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.778696 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-j5smt\"" Jul 18 01:47:41.778775 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.778726 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 18 01:47:41.779801 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.779789 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.781956 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.781861 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/270e5f34-05a1-4e34-828a-2ade34235111-hosts-file\") pod \"node-resolver-8xjzh\" (UID: \"270e5f34-05a1-4e34-828a-2ade34235111\") " pod="openshift-dns/node-resolver-8xjzh" Jul 18 01:47:41.781956 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.781882 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wj9lj" Jul 18 01:47:41.781956 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.781900 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270e5f34-05a1-4e34-828a-2ade34235111-tmp-dir\") pod \"node-resolver-8xjzh\" (UID: \"270e5f34-05a1-4e34-828a-2ade34235111\") " pod="openshift-dns/node-resolver-8xjzh" Jul 18 01:47:41.781956 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.781928 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.782189 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.781975 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-sysconfig\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.782189 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782006 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-sys\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.782189 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782032 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-lib-modules\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.782189 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782058 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-var-lib-kubelet\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.782189 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782080 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ddw6\" (UniqueName: \"kubernetes.io/projected/62155d42-0fc9-4747-a3dc-0f13a3c17f48-kube-api-access-6ddw6\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.782189 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782104 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbzfb\" (UniqueName: \"kubernetes.io/projected/270e5f34-05a1-4e34-828a-2ade34235111-kube-api-access-jbzfb\") pod \"node-resolver-8xjzh\" (UID: \"270e5f34-05a1-4e34-828a-2ade34235111\") " pod="openshift-dns/node-resolver-8xjzh" Jul 18 01:47:41.782189 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782127 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-run-ovn\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.782189 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782105 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 18 01:47:41.782189 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782168 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-run-ovn-kubernetes\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782225 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782223 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-run\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782267 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-systemd-units\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782295 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-run-netns\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782318 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-var-lib-openvswitch\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782341 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-cni-bin\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782364 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/51675a11-0cb3-4fa9-99db-3cd90c1355fa-env-overrides\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782386 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/51675a11-0cb3-4fa9-99db-3cd90c1355fa-ovnkube-script-lib\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782407 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782426 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jd5g2\"" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782426 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782411 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-sysctl-conf\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782536 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-systemd\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782561 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-tuned\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782585 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-slash\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782608 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-run-openvswitch\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782623 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-node-log\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.782667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782659 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-run-systemd\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.783344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782689 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/abf94060-f8ec-42d6-8e79-2c6d3490ebf8-host\") pod \"node-ca-566ms\" (UID: \"abf94060-f8ec-42d6-8e79-2c6d3490ebf8\") " pod="openshift-image-registry/node-ca-566ms" Jul 18 01:47:41.783344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782704 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/abf94060-f8ec-42d6-8e79-2c6d3490ebf8-serviceca\") pod \"node-ca-566ms\" (UID: \"abf94060-f8ec-42d6-8e79-2c6d3490ebf8\") " pod="openshift-image-registry/node-ca-566ms" Jul 18 01:47:41.783344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782707 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 18 01:47:41.783344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782720 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62155d42-0fc9-4747-a3dc-0f13a3c17f48-tmp\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.783344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782747 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-kubelet\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.783344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782755 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 18 01:47:41.783344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782778 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-cni-netd\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.783344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782839 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-etc-openvswitch\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.783344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782883 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/51675a11-0cb3-4fa9-99db-3cd90c1355fa-ovn-node-metrics-cert\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.783344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782920 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-sysctl-d\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.783344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782966 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-log-socket\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.783344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.782996 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/51675a11-0cb3-4fa9-99db-3cd90c1355fa-ovnkube-config\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.783344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.783022 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdfdx\" (UniqueName: \"kubernetes.io/projected/51675a11-0cb3-4fa9-99db-3cd90c1355fa-kube-api-access-pdfdx\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.783344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.783046 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-modprobe-d\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.783344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.783075 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-host\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.783344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.783111 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbsqw\" (UniqueName: \"kubernetes.io/projected/abf94060-f8ec-42d6-8e79-2c6d3490ebf8-kube-api-access-dbsqw\") pod \"node-ca-566ms\" (UID: \"abf94060-f8ec-42d6-8e79-2c6d3490ebf8\") " pod="openshift-image-registry/node-ca-566ms" Jul 18 01:47:41.783344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.783132 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-kubernetes\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.784405 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.784268 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.786370 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.786354 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 18 01:47:41.786535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.786522 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 18 01:47:41.786651 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.786623 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 18 01:47:41.786719 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.786705 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hvdg2\"" Jul 18 01:47:41.786771 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.786722 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.786991 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.786955 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-stdrl\"" Jul 18 01:47:41.787086 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.786994 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 18 01:47:41.787166 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.787152 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 18 01:47:41.788730 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.788712 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 18 01:47:41.788974 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.788934 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 18 01:47:41.788974 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.788937 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 18 01:47:41.789157 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.789141 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 18 01:47:41.789326 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.789306 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 18 01:47:41.789457 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.789387 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gldf4\"" Jul 18 01:47:41.789457 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.789399 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.789672 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.789618 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-17 01:42:40 +0000 UTC" deadline="2028-03-29 10:52:32.42215683 +0000 UTC" Jul 18 01:47:41.789714 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.789673 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14889h4m50.632486659s" Jul 18 01:47:41.791154 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.791137 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rj2mt\"" Jul 18 01:47:41.791325 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.791314 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 18 01:47:41.792475 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.792460 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:41.792579 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:41.792559 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:47:41.794839 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.794822 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 01:47:41.795238 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.795209 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:47:41.795328 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:41.795281 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:47:41.797345 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.797321 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 01:47:41.797449 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.797409 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" Jul 18 01:47:41.799926 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.799905 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kj7rk" Jul 18 01:47:41.801882 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.801864 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lvbch\"" Jul 18 01:47:41.802089 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.802064 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 18 01:47:41.802191 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.802091 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 18 01:47:41.802191 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.802146 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 18 01:47:41.802408 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.802392 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal"] Jul 18 01:47:41.802470 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.802459 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:41.802524 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:41.802510 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:47:41.805672 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.805655 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 01:47:41.805895 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.805881 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal"] Jul 18 01:47:41.820393 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.820373 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-tf9zj" Jul 18 01:47:41.826367 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.826350 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-tf9zj" Jul 18 01:47:41.832563 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.832541 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:47:41.877916 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.877896 2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 18 01:47:41.883376 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883356 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-sys\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.883477 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883388 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-lib-modules\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.883477 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883413 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-var-lib-kubelet\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.883477 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883435 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ddw6\" (UniqueName: \"kubernetes.io/projected/62155d42-0fc9-4747-a3dc-0f13a3c17f48-kube-api-access-6ddw6\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.883477 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883459 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbzfb\" (UniqueName: \"kubernetes.io/projected/270e5f34-05a1-4e34-828a-2ade34235111-kube-api-access-jbzfb\") pod \"node-resolver-8xjzh\" (UID: \"270e5f34-05a1-4e34-828a-2ade34235111\") " pod="openshift-dns/node-resolver-8xjzh" Jul 18 01:47:41.883692 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883476 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-sys\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.883692 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883485 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-var-lib-kubelet\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.883692 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883480 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-run-ovn\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.883692 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883516 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-run-ovn\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.883692 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883517 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-lib-modules\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.883692 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883530 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-system-cni-dir\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.883692 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883558 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgqq2\" (UniqueName: \"kubernetes.io/projected/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-kube-api-access-jgqq2\") pod \"network-metrics-daemon-5nqdq\" (UID: \"bf7391f7-3b4a-4eb5-ab3a-b540543d4907\") " pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:41.883692 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883584 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-systemd-units\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.883692 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883612 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-run-netns\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.883692 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883661 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-var-lib-openvswitch\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.883692 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883679 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-systemd-units\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.884165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883704 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-cni-bin\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.884165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883689 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-run-netns\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.884165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883722 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-var-lib-openvswitch\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.884165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883752 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-cni-bin\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.884165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883772 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/51675a11-0cb3-4fa9-99db-3cd90c1355fa-ovnkube-script-lib\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.884165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883819 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-var-lib-cni-multus\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.884165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883846 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-hostroot\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.884165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883873 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8tp6\" (UniqueName: \"kubernetes.io/projected/05401e73-f450-4557-b68f-5092e860981c-kube-api-access-s8tp6\") pod \"iptables-alerter-kj7rk\" (UID: \"05401e73-f450-4557-b68f-5092e860981c\") " pod="openshift-network-operator/iptables-alerter-kj7rk" Jul 18 01:47:41.884165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883896 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-sysctl-conf\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.884165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883918 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-systemd\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.884165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883941 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-tuned\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.884165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883963 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-slash\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.884165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.883987 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-run-openvswitch\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.884165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884012 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-node-log\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.884165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884050 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d53a7743-60f0-4308-8165-6b27513efeb4-cni-binary-copy\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.884165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884054 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-sysctl-conf\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.884165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884073 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62155d42-0fc9-4747-a3dc-0f13a3c17f48-tmp\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.885047 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884097 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-kubelet\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.885047 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884108 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-slash\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.885047 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884119 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-cni-netd\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.885047 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884143 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d53a7743-60f0-4308-8165-6b27513efeb4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.885047 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884159 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-systemd\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.885047 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884167 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-kubelet-config\") pod \"global-pull-secret-syncer-bdkrt\" (UID: \"2693ad6a-f241-4165-8d2e-9aee83cc8ae8\") " pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:41.885047 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884198 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-cnibin\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.885047 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884213 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/51675a11-0cb3-4fa9-99db-3cd90c1355fa-ovnkube-script-lib\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.885047 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884221 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs\") pod \"network-metrics-daemon-5nqdq\" (UID: \"bf7391f7-3b4a-4eb5-ab3a-b540543d4907\") " pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:41.885047 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884255 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/05401e73-f450-4557-b68f-5092e860981c-host-slash\") pod \"iptables-alerter-kj7rk\" (UID: \"05401e73-f450-4557-b68f-5092e860981c\") " pod="openshift-network-operator/iptables-alerter-kj7rk" Jul 18 01:47:41.885047 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884294 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-node-log\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.885047 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884309 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-cni-netd\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.885047 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884342 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-run-openvswitch\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.885047 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884380 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-kubelet\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.885047 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884390 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/51675a11-0cb3-4fa9-99db-3cd90c1355fa-ovn-node-metrics-cert\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.885047 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884423 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8baf11f1-2c27-4a7c-97c1-73f26856fe39-multus-daemon-config\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.885047 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884472 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-log-socket\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.885758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884509 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdfdx\" (UniqueName: \"kubernetes.io/projected/51675a11-0cb3-4fa9-99db-3cd90c1355fa-kube-api-access-pdfdx\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.885758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884532 2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 18 01:47:41.885758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884544 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-log-socket\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.885758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884563 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d53a7743-60f0-4308-8165-6b27513efeb4-cnibin\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.885758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884590 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-host\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.885758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884614 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-socket-dir\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.885758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884674 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-device-dir\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.885758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884708 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-host\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.885758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884750 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm4cp\" (UniqueName: \"kubernetes.io/projected/d287d9c6-2390-44b3-902b-3cf24d7a32e0-kube-api-access-bm4cp\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.885758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884776 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d53a7743-60f0-4308-8165-6b27513efeb4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.885758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884821 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8baf11f1-2c27-4a7c-97c1-73f26856fe39-cni-binary-copy\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.885758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884848 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-run-netns\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.885758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884869 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-multus-conf-dir\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.885758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884896 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-kubernetes\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.885758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884922 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-multus-cni-dir\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.885758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884958 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-kubernetes\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.885758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884971 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-run-k8s-cni-cncf-io\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.886535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.884998 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-run-multus-certs\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.886535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885015 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-sysconfig\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.886535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885031 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-run-ovn-kubernetes\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.886535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885048 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/51675a11-0cb3-4fa9-99db-3cd90c1355fa-env-overrides\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.886535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885072 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.886535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885089 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-sysconfig\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.886535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885095 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-os-release\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.886535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885097 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-run-ovn-kubernetes\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.886535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885147 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-etc-kubernetes\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.886535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885177 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-run\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.886535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885206 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74h8p\" (UniqueName: \"kubernetes.io/projected/d53a7743-60f0-4308-8165-6b27513efeb4-kube-api-access-74h8p\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.886535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885228 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-multus-socket-dir-parent\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.886535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885275 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-var-lib-cni-bin\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.886535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885315 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-sys-fs\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.886535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885338 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d53a7743-60f0-4308-8165-6b27513efeb4-system-cni-dir\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.886535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885375 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d53a7743-60f0-4308-8165-6b27513efeb4-os-release\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.886535 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885402 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/51675a11-0cb3-4fa9-99db-3cd90c1355fa-env-overrides\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.887343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885400 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwchk\" (UniqueName: \"kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk\") pod \"network-check-target-2shnl\" (UID: \"af86de24-1529-4faa-9cdb-48c4cf6d6ede\") " pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:47:41.887343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885442 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-run\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.887343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885450 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret\") pod \"global-pull-secret-syncer-bdkrt\" (UID: \"2693ad6a-f241-4165-8d2e-9aee83cc8ae8\") " pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:41.887343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885478 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-run-systemd\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.887343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885505 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/abf94060-f8ec-42d6-8e79-2c6d3490ebf8-host\") pod \"node-ca-566ms\" (UID: \"abf94060-f8ec-42d6-8e79-2c6d3490ebf8\") " pod="openshift-image-registry/node-ca-566ms" Jul 18 01:47:41.887343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885522 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/abf94060-f8ec-42d6-8e79-2c6d3490ebf8-serviceca\") pod \"node-ca-566ms\" (UID: \"abf94060-f8ec-42d6-8e79-2c6d3490ebf8\") " pod="openshift-image-registry/node-ca-566ms" Jul 18 01:47:41.887343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885528 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-run-systemd\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.887343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885548 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-etc-selinux\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.887343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885564 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-var-lib-kubelet\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.887343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885568 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/abf94060-f8ec-42d6-8e79-2c6d3490ebf8-host\") pod \"node-ca-566ms\" (UID: \"abf94060-f8ec-42d6-8e79-2c6d3490ebf8\") " pod="openshift-image-registry/node-ca-566ms" Jul 18 01:47:41.887343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885580 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-etc-openvswitch\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.887343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885606 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/05401e73-f450-4557-b68f-5092e860981c-iptables-alerter-script\") pod \"iptables-alerter-kj7rk\" (UID: \"05401e73-f450-4557-b68f-5092e860981c\") " pod="openshift-network-operator/iptables-alerter-kj7rk" Jul 18 01:47:41.887343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885656 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-etc-openvswitch\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.887343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885685 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-sysctl-d\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.887343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885708 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/51675a11-0cb3-4fa9-99db-3cd90c1355fa-ovnkube-config\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.887343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885750 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-registration-dir\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.887343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885780 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d53a7743-60f0-4308-8165-6b27513efeb4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.888103 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885806 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-modprobe-d\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.888103 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885884 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-modprobe-d\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.888103 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.885956 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-sysctl-d\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.888103 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.886104 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/abf94060-f8ec-42d6-8e79-2c6d3490ebf8-serviceca\") pod \"node-ca-566ms\" (UID: \"abf94060-f8ec-42d6-8e79-2c6d3490ebf8\") " pod="openshift-image-registry/node-ca-566ms" Jul 18 01:47:41.888103 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.886107 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8af123e0-fca5-4d69-b7cd-dcfdc8e1750e-agent-certs\") pod \"konnectivity-agent-wj9lj\" (UID: \"8af123e0-fca5-4d69-b7cd-dcfdc8e1750e\") " pod="kube-system/konnectivity-agent-wj9lj" Jul 18 01:47:41.888103 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.886163 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbsqw\" (UniqueName: \"kubernetes.io/projected/abf94060-f8ec-42d6-8e79-2c6d3490ebf8-kube-api-access-dbsqw\") pod \"node-ca-566ms\" (UID: \"abf94060-f8ec-42d6-8e79-2c6d3490ebf8\") " pod="openshift-image-registry/node-ca-566ms" Jul 18 01:47:41.888103 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.886189 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/270e5f34-05a1-4e34-828a-2ade34235111-hosts-file\") pod \"node-resolver-8xjzh\" (UID: \"270e5f34-05a1-4e34-828a-2ade34235111\") " pod="openshift-dns/node-resolver-8xjzh" Jul 18 01:47:41.888103 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.886214 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270e5f34-05a1-4e34-828a-2ade34235111-tmp-dir\") pod \"node-resolver-8xjzh\" (UID: \"270e5f34-05a1-4e34-828a-2ade34235111\") " pod="openshift-dns/node-resolver-8xjzh" Jul 18 01:47:41.888103 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.886239 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.888103 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.886267 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-dbus\") pod \"global-pull-secret-syncer-bdkrt\" (UID: \"2693ad6a-f241-4165-8d2e-9aee83cc8ae8\") " pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:41.888103 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.886285 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/51675a11-0cb3-4fa9-99db-3cd90c1355fa-ovnkube-config\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.888103 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.886289 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/270e5f34-05a1-4e34-828a-2ade34235111-hosts-file\") pod \"node-resolver-8xjzh\" (UID: \"270e5f34-05a1-4e34-828a-2ade34235111\") " pod="openshift-dns/node-resolver-8xjzh" Jul 18 01:47:41.888103 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.886292 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8af123e0-fca5-4d69-b7cd-dcfdc8e1750e-konnectivity-ca\") pod \"konnectivity-agent-wj9lj\" (UID: \"8af123e0-fca5-4d69-b7cd-dcfdc8e1750e\") " pod="kube-system/konnectivity-agent-wj9lj" Jul 18 01:47:41.888103 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.886341 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd2p8\" (UniqueName: \"kubernetes.io/projected/8baf11f1-2c27-4a7c-97c1-73f26856fe39-kube-api-access-sd2p8\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.888103 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.886424 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51675a11-0cb3-4fa9-99db-3cd90c1355fa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.888103 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.886628 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/270e5f34-05a1-4e34-828a-2ade34235111-tmp-dir\") pod \"node-resolver-8xjzh\" (UID: \"270e5f34-05a1-4e34-828a-2ade34235111\") " pod="openshift-dns/node-resolver-8xjzh" Jul 18 01:47:41.888556 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.888143 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/62155d42-0fc9-4747-a3dc-0f13a3c17f48-tmp\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.888556 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.888147 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/62155d42-0fc9-4747-a3dc-0f13a3c17f48-etc-tuned\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.888556 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.888159 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/51675a11-0cb3-4fa9-99db-3cd90c1355fa-ovn-node-metrics-cert\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.900580 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.900549 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdfdx\" (UniqueName: \"kubernetes.io/projected/51675a11-0cb3-4fa9-99db-3cd90c1355fa-kube-api-access-pdfdx\") pod \"ovnkube-node-jshl6\" (UID: \"51675a11-0cb3-4fa9-99db-3cd90c1355fa\") " pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:41.900680 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.900612 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbsqw\" (UniqueName: \"kubernetes.io/projected/abf94060-f8ec-42d6-8e79-2c6d3490ebf8-kube-api-access-dbsqw\") pod \"node-ca-566ms\" (UID: \"abf94060-f8ec-42d6-8e79-2c6d3490ebf8\") " pod="openshift-image-registry/node-ca-566ms" Jul 18 01:47:41.901230 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.901214 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ddw6\" (UniqueName: \"kubernetes.io/projected/62155d42-0fc9-4747-a3dc-0f13a3c17f48-kube-api-access-6ddw6\") pod \"tuned-w5hwx\" (UID: \"62155d42-0fc9-4747-a3dc-0f13a3c17f48\") " pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:41.903945 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.903926 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbzfb\" (UniqueName: \"kubernetes.io/projected/270e5f34-05a1-4e34-828a-2ade34235111-kube-api-access-jbzfb\") pod \"node-resolver-8xjzh\" (UID: \"270e5f34-05a1-4e34-828a-2ade34235111\") " pod="openshift-dns/node-resolver-8xjzh" Jul 18 01:47:41.973592 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:41.973548 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90ea7be93df65d66da652d9417af881e.slice/crio-0cc6ed5d62f261a82b6580ed2298961cb6b18832302e174f74ae1f1f0958b576 WatchSource:0}: Error finding container 0cc6ed5d62f261a82b6580ed2298961cb6b18832302e174f74ae1f1f0958b576: Status 404 returned error can't find the container with id 0cc6ed5d62f261a82b6580ed2298961cb6b18832302e174f74ae1f1f0958b576 Jul 18 01:47:41.974409 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:41.974386 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15fcf4b6dde762c7ce3ec0cc2a1506cd.slice/crio-f0676280a6d9209768a5293f29ad49a1615b1adcd1a9cde40efb71e29bc63989 WatchSource:0}: Error finding container f0676280a6d9209768a5293f29ad49a1615b1adcd1a9cde40efb71e29bc63989: Status 404 returned error can't find the container with id f0676280a6d9209768a5293f29ad49a1615b1adcd1a9cde40efb71e29bc63989 Jul 18 01:47:41.979090 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.979073 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 01:47:41.987522 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987499 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/05401e73-f450-4557-b68f-5092e860981c-iptables-alerter-script\") pod \"iptables-alerter-kj7rk\" (UID: \"05401e73-f450-4557-b68f-5092e860981c\") " pod="openshift-network-operator/iptables-alerter-kj7rk" Jul 18 01:47:41.987627 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987528 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-registration-dir\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.987627 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987545 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d53a7743-60f0-4308-8165-6b27513efeb4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.987627 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987564 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8af123e0-fca5-4d69-b7cd-dcfdc8e1750e-agent-certs\") pod \"konnectivity-agent-wj9lj\" (UID: \"8af123e0-fca5-4d69-b7cd-dcfdc8e1750e\") " pod="kube-system/konnectivity-agent-wj9lj" Jul 18 01:47:41.987627 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987587 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-dbus\") pod \"global-pull-secret-syncer-bdkrt\" (UID: \"2693ad6a-f241-4165-8d2e-9aee83cc8ae8\") " pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:41.987862 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987627 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-registration-dir\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.987862 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987636 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8af123e0-fca5-4d69-b7cd-dcfdc8e1750e-konnectivity-ca\") pod \"konnectivity-agent-wj9lj\" (UID: \"8af123e0-fca5-4d69-b7cd-dcfdc8e1750e\") " pod="kube-system/konnectivity-agent-wj9lj" Jul 18 01:47:41.987862 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987688 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd2p8\" (UniqueName: \"kubernetes.io/projected/8baf11f1-2c27-4a7c-97c1-73f26856fe39-kube-api-access-sd2p8\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.987862 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987722 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-system-cni-dir\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.987862 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987742 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-dbus\") pod \"global-pull-secret-syncer-bdkrt\" (UID: \"2693ad6a-f241-4165-8d2e-9aee83cc8ae8\") " pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:41.987862 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987749 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgqq2\" (UniqueName: \"kubernetes.io/projected/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-kube-api-access-jgqq2\") pod \"network-metrics-daemon-5nqdq\" (UID: \"bf7391f7-3b4a-4eb5-ab3a-b540543d4907\") " pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:41.987862 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987781 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-var-lib-cni-multus\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.987862 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987807 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-hostroot\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.987862 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987831 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8tp6\" (UniqueName: \"kubernetes.io/projected/05401e73-f450-4557-b68f-5092e860981c-kube-api-access-s8tp6\") pod \"iptables-alerter-kj7rk\" (UID: \"05401e73-f450-4557-b68f-5092e860981c\") " pod="openshift-network-operator/iptables-alerter-kj7rk" Jul 18 01:47:41.987862 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987849 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-var-lib-cni-multus\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.987862 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987862 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d53a7743-60f0-4308-8165-6b27513efeb4-cni-binary-copy\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.987862 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987860 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-hostroot\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.988420 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987918 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d53a7743-60f0-4308-8165-6b27513efeb4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.988420 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987943 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-kubelet-config\") pod \"global-pull-secret-syncer-bdkrt\" (UID: \"2693ad6a-f241-4165-8d2e-9aee83cc8ae8\") " pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:41.988420 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.987984 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-cnibin\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.988420 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988018 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs\") pod \"network-metrics-daemon-5nqdq\" (UID: \"bf7391f7-3b4a-4eb5-ab3a-b540543d4907\") " pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:41.988420 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988033 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/05401e73-f450-4557-b68f-5092e860981c-host-slash\") pod \"iptables-alerter-kj7rk\" (UID: \"05401e73-f450-4557-b68f-5092e860981c\") " pod="openshift-network-operator/iptables-alerter-kj7rk" Jul 18 01:47:41.988420 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988077 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/05401e73-f450-4557-b68f-5092e860981c-host-slash\") pod \"iptables-alerter-kj7rk\" (UID: \"05401e73-f450-4557-b68f-5092e860981c\") " pod="openshift-network-operator/iptables-alerter-kj7rk" Jul 18 01:47:41.988420 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988111 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-kubelet-config\") pod \"global-pull-secret-syncer-bdkrt\" (UID: \"2693ad6a-f241-4165-8d2e-9aee83cc8ae8\") " pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:41.988420 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988111 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-system-cni-dir\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.988420 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988151 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/05401e73-f450-4557-b68f-5092e860981c-iptables-alerter-script\") pod \"iptables-alerter-kj7rk\" (UID: \"05401e73-f450-4557-b68f-5092e860981c\") " pod="openshift-network-operator/iptables-alerter-kj7rk" Jul 18 01:47:41.988420 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988161 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-cnibin\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.988420 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:41.988177 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:47:41.988420 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988174 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d53a7743-60f0-4308-8165-6b27513efeb4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.988420 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988188 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8baf11f1-2c27-4a7c-97c1-73f26856fe39-multus-daemon-config\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.988420 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:41.988228 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs podName:bf7391f7-3b4a-4eb5-ab3a-b540543d4907 nodeName:}" failed. No retries permitted until 2026-07-18 01:47:42.48821233 +0000 UTC m=+2.165636434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs") pod "network-metrics-daemon-5nqdq" (UID: "bf7391f7-3b4a-4eb5-ab3a-b540543d4907") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:47:41.988420 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988254 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d53a7743-60f0-4308-8165-6b27513efeb4-cnibin\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.988420 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988294 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-socket-dir\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.988420 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988302 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d53a7743-60f0-4308-8165-6b27513efeb4-cnibin\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.989213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988316 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-device-dir\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.989213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988348 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm4cp\" (UniqueName: \"kubernetes.io/projected/d287d9c6-2390-44b3-902b-3cf24d7a32e0-kube-api-access-bm4cp\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.989213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988371 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d53a7743-60f0-4308-8165-6b27513efeb4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.989213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988392 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8baf11f1-2c27-4a7c-97c1-73f26856fe39-cni-binary-copy\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.989213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988415 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-run-netns\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.989213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988420 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d53a7743-60f0-4308-8165-6b27513efeb4-cni-binary-copy\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.989213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988439 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-multus-conf-dir\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.989213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988452 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d53a7743-60f0-4308-8165-6b27513efeb4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.989213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988468 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-multus-cni-dir\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.989213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988492 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-run-k8s-cni-cncf-io\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.989213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988514 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-run-multus-certs\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.989213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988542 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.989213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988563 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-os-release\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.989213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988423 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-socket-dir\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.989213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988594 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-etc-kubernetes\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.989213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988620 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74h8p\" (UniqueName: \"kubernetes.io/projected/d53a7743-60f0-4308-8165-6b27513efeb4-kube-api-access-74h8p\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.989213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988666 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-multus-socket-dir-parent\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.989774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988689 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-var-lib-cni-bin\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.989774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988701 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-run-multus-certs\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.989774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988710 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-sys-fs\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.989774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988745 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d53a7743-60f0-4308-8165-6b27513efeb4-system-cni-dir\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.989774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988768 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-sys-fs\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.989774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988769 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8baf11f1-2c27-4a7c-97c1-73f26856fe39-multus-daemon-config\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.989774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988621 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d53a7743-60f0-4308-8165-6b27513efeb4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.989774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988470 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-device-dir\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.989774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988830 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-run-k8s-cni-cncf-io\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.989774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988838 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-run-netns\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.989774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988827 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8af123e0-fca5-4d69-b7cd-dcfdc8e1750e-konnectivity-ca\") pod \"konnectivity-agent-wj9lj\" (UID: \"8af123e0-fca5-4d69-b7cd-dcfdc8e1750e\") " pod="kube-system/konnectivity-agent-wj9lj" Jul 18 01:47:41.989774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988776 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d53a7743-60f0-4308-8165-6b27513efeb4-os-release\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.989774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988842 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-multus-conf-dir\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.989774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988842 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-os-release\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.989774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988880 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-etc-kubernetes\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.989774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988883 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwchk\" (UniqueName: \"kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk\") pod \"network-check-target-2shnl\" (UID: \"af86de24-1529-4faa-9cdb-48c4cf6d6ede\") " pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:47:41.989774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988900 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d53a7743-60f0-4308-8165-6b27513efeb4-os-release\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.990251 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988912 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret\") pod \"global-pull-secret-syncer-bdkrt\" (UID: \"2693ad6a-f241-4165-8d2e-9aee83cc8ae8\") " pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:41.990251 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988934 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.990251 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988939 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-var-lib-cni-bin\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.990251 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988944 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-etc-selinux\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.990251 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988968 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-var-lib-kubelet\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.990251 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988992 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-multus-cni-dir\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.990251 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988993 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d53a7743-60f0-4308-8165-6b27513efeb4-system-cni-dir\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.990251 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.988887 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-multus-socket-dir-parent\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.990251 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.989041 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8baf11f1-2c27-4a7c-97c1-73f26856fe39-host-var-lib-kubelet\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.990251 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.989115 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8baf11f1-2c27-4a7c-97c1-73f26856fe39-cni-binary-copy\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:41.990251 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.989137 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d287d9c6-2390-44b3-902b-3cf24d7a32e0-etc-selinux\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.990251 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:41.989141 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:47:41.990251 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:41.989213 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret podName:2693ad6a-f241-4165-8d2e-9aee83cc8ae8 nodeName:}" failed. No retries permitted until 2026-07-18 01:47:42.489194912 +0000 UTC m=+2.166619013 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret") pod "global-pull-secret-syncer-bdkrt" (UID: "2693ad6a-f241-4165-8d2e-9aee83cc8ae8") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:47:41.990955 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.990937 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8af123e0-fca5-4d69-b7cd-dcfdc8e1750e-agent-certs\") pod \"konnectivity-agent-wj9lj\" (UID: \"8af123e0-fca5-4d69-b7cd-dcfdc8e1750e\") " pod="kube-system/konnectivity-agent-wj9lj" Jul 18 01:47:41.995020 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:41.995000 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:47:41.995106 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:41.995025 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:47:41.995106 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:41.995041 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qwchk for pod openshift-network-diagnostics/network-check-target-2shnl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:47:41.995208 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:41.995105 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk podName:af86de24-1529-4faa-9cdb-48c4cf6d6ede nodeName:}" failed. No retries permitted until 2026-07-18 01:47:42.495086902 +0000 UTC m=+2.172511016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-qwchk" (UniqueName: "kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk") pod "network-check-target-2shnl" (UID: "af86de24-1529-4faa-9cdb-48c4cf6d6ede") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:47:41.996482 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.996451 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8tp6\" (UniqueName: \"kubernetes.io/projected/05401e73-f450-4557-b68f-5092e860981c-kube-api-access-s8tp6\") pod \"iptables-alerter-kj7rk\" (UID: \"05401e73-f450-4557-b68f-5092e860981c\") " pod="openshift-network-operator/iptables-alerter-kj7rk" Jul 18 01:47:41.996570 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.996457 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgqq2\" (UniqueName: \"kubernetes.io/projected/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-kube-api-access-jgqq2\") pod \"network-metrics-daemon-5nqdq\" (UID: \"bf7391f7-3b4a-4eb5-ab3a-b540543d4907\") " pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:41.996935 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.996915 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74h8p\" (UniqueName: \"kubernetes.io/projected/d53a7743-60f0-4308-8165-6b27513efeb4-kube-api-access-74h8p\") pod \"multus-additional-cni-plugins-5lxc9\" (UID: \"d53a7743-60f0-4308-8165-6b27513efeb4\") " pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:41.997092 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.997075 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm4cp\" (UniqueName: \"kubernetes.io/projected/d287d9c6-2390-44b3-902b-3cf24d7a32e0-kube-api-access-bm4cp\") pod \"aws-ebs-csi-driver-node-fkckl\" (UID: \"d287d9c6-2390-44b3-902b-3cf24d7a32e0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:41.997157 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:41.997137 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd2p8\" (UniqueName: \"kubernetes.io/projected/8baf11f1-2c27-4a7c-97c1-73f26856fe39-kube-api-access-sd2p8\") pod \"multus-xbkpg\" (UID: \"8baf11f1-2c27-4a7c-97c1-73f26856fe39\") " pod="openshift-multus/multus-xbkpg" Jul 18 01:47:42.104311 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.104278 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-566ms" Jul 18 01:47:42.113945 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:42.113918 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabf94060_f8ec_42d6_8e79_2c6d3490ebf8.slice/crio-5f76bdfb4f8fd5cbf21d605aa2c0530963674129c9f910367e883ba7c7ecb87f WatchSource:0}: Error finding container 5f76bdfb4f8fd5cbf21d605aa2c0530963674129c9f910367e883ba7c7ecb87f: Status 404 returned error can't find the container with id 5f76bdfb4f8fd5cbf21d605aa2c0530963674129c9f910367e883ba7c7ecb87f Jul 18 01:47:42.118025 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.118006 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" Jul 18 01:47:42.125349 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:42.125321 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62155d42_0fc9_4747_a3dc_0f13a3c17f48.slice/crio-d5eb7313a56c3da1d2ab69e75f40a75e72963cab96e2df2025df678dc969ae55 WatchSource:0}: Error finding container d5eb7313a56c3da1d2ab69e75f40a75e72963cab96e2df2025df678dc969ae55: Status 404 returned error can't find the container with id d5eb7313a56c3da1d2ab69e75f40a75e72963cab96e2df2025df678dc969ae55 Jul 18 01:47:42.137008 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.136981 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8xjzh" Jul 18 01:47:42.144901 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:42.144878 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod270e5f34_05a1_4e34_828a_2ade34235111.slice/crio-50eecfb11b2b62822b1dbeb789495f40067f24a81fa6a218ecea256860f9fa4c WatchSource:0}: Error finding container 50eecfb11b2b62822b1dbeb789495f40067f24a81fa6a218ecea256860f9fa4c: Status 404 returned error can't find the container with id 50eecfb11b2b62822b1dbeb789495f40067f24a81fa6a218ecea256860f9fa4c Jul 18 01:47:42.157282 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.157266 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:47:42.162102 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.162087 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wj9lj" Jul 18 01:47:42.165908 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:42.165886 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51675a11_0cb3_4fa9_99db_3cd90c1355fa.slice/crio-9895b110b5774890db55e84194cb219e688813971c504052422d74af9bc38d84 WatchSource:0}: Error finding container 9895b110b5774890db55e84194cb219e688813971c504052422d74af9bc38d84: Status 404 returned error can't find the container with id 9895b110b5774890db55e84194cb219e688813971c504052422d74af9bc38d84 Jul 18 01:47:42.170541 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:42.170520 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8af123e0_fca5_4d69_b7cd_dcfdc8e1750e.slice/crio-ac42ca8e38468f523e09937baba1463d1102da325448ce3945f0d56030842164 WatchSource:0}: Error finding container ac42ca8e38468f523e09937baba1463d1102da325448ce3945f0d56030842164: Status 404 returned error can't find the container with id ac42ca8e38468f523e09937baba1463d1102da325448ce3945f0d56030842164 Jul 18 01:47:42.178862 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.178846 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" Jul 18 01:47:42.189039 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:42.189018 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd287d9c6_2390_44b3_902b_3cf24d7a32e0.slice/crio-0cd4ee27cc54b2aceb97201c2af2fb2b31b575dfeafc104c412ea5839dd480ea WatchSource:0}: Error finding container 0cd4ee27cc54b2aceb97201c2af2fb2b31b575dfeafc104c412ea5839dd480ea: Status 404 returned error can't find the container with id 0cd4ee27cc54b2aceb97201c2af2fb2b31b575dfeafc104c412ea5839dd480ea Jul 18 01:47:42.199061 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.199042 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5lxc9" Jul 18 01:47:42.199875 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.199855 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:47:42.206903 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:42.206886 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd53a7743_60f0_4308_8165_6b27513efeb4.slice/crio-d6ac4cb782e0b4e13acdb338786ab666fb46dad638be4dc00a4b30ad1884e9d9 WatchSource:0}: Error finding container d6ac4cb782e0b4e13acdb338786ab666fb46dad638be4dc00a4b30ad1884e9d9: Status 404 returned error can't find the container with id d6ac4cb782e0b4e13acdb338786ab666fb46dad638be4dc00a4b30ad1884e9d9 Jul 18 01:47:42.220702 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.220658 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xbkpg" Jul 18 01:47:42.225440 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.225424 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-kj7rk" Jul 18 01:47:42.228898 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:42.228878 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8baf11f1_2c27_4a7c_97c1_73f26856fe39.slice/crio-33bd567454e3390405570247e5d7e69d4c11ef07a53a0444c3f6d2d22fa5cef6 WatchSource:0}: Error finding container 33bd567454e3390405570247e5d7e69d4c11ef07a53a0444c3f6d2d22fa5cef6: Status 404 returned error can't find the container with id 33bd567454e3390405570247e5d7e69d4c11ef07a53a0444c3f6d2d22fa5cef6 Jul 18 01:47:42.233422 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:47:42.233396 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05401e73_f450_4557_b68f_5092e860981c.slice/crio-ba30b9bd9de129e72aa45352575d98f688f74ad8c1c942e088896eaa3647a1a1 WatchSource:0}: Error finding container ba30b9bd9de129e72aa45352575d98f688f74ad8c1c942e088896eaa3647a1a1: Status 404 returned error can't find the container with id ba30b9bd9de129e72aa45352575d98f688f74ad8c1c942e088896eaa3647a1a1 Jul 18 01:47:42.492121 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.492029 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret\") pod \"global-pull-secret-syncer-bdkrt\" (UID: \"2693ad6a-f241-4165-8d2e-9aee83cc8ae8\") " pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:42.492121 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.492081 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs\") pod \"network-metrics-daemon-5nqdq\" (UID: \"bf7391f7-3b4a-4eb5-ab3a-b540543d4907\") " pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:42.492327 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:42.492193 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:47:42.492327 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:42.492212 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:47:42.492327 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:42.492259 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs podName:bf7391f7-3b4a-4eb5-ab3a-b540543d4907 nodeName:}" failed. No retries permitted until 2026-07-18 01:47:43.492242925 +0000 UTC m=+3.169667013 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs") pod "network-metrics-daemon-5nqdq" (UID: "bf7391f7-3b4a-4eb5-ab3a-b540543d4907") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:47:42.492327 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:42.492276 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret podName:2693ad6a-f241-4165-8d2e-9aee83cc8ae8 nodeName:}" failed. No retries permitted until 2026-07-18 01:47:43.492268192 +0000 UTC m=+3.169692287 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret") pod "global-pull-secret-syncer-bdkrt" (UID: "2693ad6a-f241-4165-8d2e-9aee83cc8ae8") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:47:42.592952 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.592912 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwchk\" (UniqueName: \"kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk\") pod \"network-check-target-2shnl\" (UID: \"af86de24-1529-4faa-9cdb-48c4cf6d6ede\") " pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:47:42.593127 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:42.593082 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:47:42.593127 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:42.593104 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:47:42.593127 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:42.593113 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qwchk for pod openshift-network-diagnostics/network-check-target-2shnl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:47:42.593288 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:42.593171 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk podName:af86de24-1529-4faa-9cdb-48c4cf6d6ede nodeName:}" failed. No retries permitted until 2026-07-18 01:47:43.593149444 +0000 UTC m=+3.270573545 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwchk" (UniqueName: "kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk") pod "network-check-target-2shnl" (UID: "af86de24-1529-4faa-9cdb-48c4cf6d6ede") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:47:42.829888 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.829443 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:42:41 +0000 UTC" deadline="2028-04-01 21:17:54.685324638 +0000 UTC" Jul 18 01:47:42.829888 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.829514 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14971h30m11.855815609s" Jul 18 01:47:42.910353 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.910261 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kj7rk" event={"ID":"05401e73-f450-4557-b68f-5092e860981c","Type":"ContainerStarted","Data":"ba30b9bd9de129e72aa45352575d98f688f74ad8c1c942e088896eaa3647a1a1"} Jul 18 01:47:42.930191 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.930144 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5lxc9" event={"ID":"d53a7743-60f0-4308-8165-6b27513efeb4","Type":"ContainerStarted","Data":"d6ac4cb782e0b4e13acdb338786ab666fb46dad638be4dc00a4b30ad1884e9d9"} Jul 18 01:47:42.936748 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.936712 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" event={"ID":"d287d9c6-2390-44b3-902b-3cf24d7a32e0","Type":"ContainerStarted","Data":"0cd4ee27cc54b2aceb97201c2af2fb2b31b575dfeafc104c412ea5839dd480ea"} Jul 18 01:47:42.958477 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.958432 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wj9lj" event={"ID":"8af123e0-fca5-4d69-b7cd-dcfdc8e1750e","Type":"ContainerStarted","Data":"ac42ca8e38468f523e09937baba1463d1102da325448ce3945f0d56030842164"} Jul 18 01:47:42.985814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:42.985771 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8xjzh" event={"ID":"270e5f34-05a1-4e34-828a-2ade34235111","Type":"ContainerStarted","Data":"50eecfb11b2b62822b1dbeb789495f40067f24a81fa6a218ecea256860f9fa4c"} Jul 18 01:47:43.001601 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:43.001566 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-566ms" event={"ID":"abf94060-f8ec-42d6-8e79-2c6d3490ebf8","Type":"ContainerStarted","Data":"5f76bdfb4f8fd5cbf21d605aa2c0530963674129c9f910367e883ba7c7ecb87f"} Jul 18 01:47:43.031412 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:43.031219 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" event={"ID":"15fcf4b6dde762c7ce3ec0cc2a1506cd","Type":"ContainerStarted","Data":"f0676280a6d9209768a5293f29ad49a1615b1adcd1a9cde40efb71e29bc63989"} Jul 18 01:47:43.053608 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:43.053570 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xbkpg" event={"ID":"8baf11f1-2c27-4a7c-97c1-73f26856fe39","Type":"ContainerStarted","Data":"33bd567454e3390405570247e5d7e69d4c11ef07a53a0444c3f6d2d22fa5cef6"} Jul 18 01:47:43.067088 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:43.066886 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" event={"ID":"51675a11-0cb3-4fa9-99db-3cd90c1355fa","Type":"ContainerStarted","Data":"9895b110b5774890db55e84194cb219e688813971c504052422d74af9bc38d84"} Jul 18 01:47:43.096003 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:43.095532 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" event={"ID":"62155d42-0fc9-4747-a3dc-0f13a3c17f48","Type":"ContainerStarted","Data":"d5eb7313a56c3da1d2ab69e75f40a75e72963cab96e2df2025df678dc969ae55"} Jul 18 01:47:43.118961 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:43.118926 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" event={"ID":"90ea7be93df65d66da652d9417af881e","Type":"ContainerStarted","Data":"0cc6ed5d62f261a82b6580ed2298961cb6b18832302e174f74ae1f1f0958b576"} Jul 18 01:47:43.144425 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:43.144396 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:47:43.507615 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:43.507532 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret\") pod \"global-pull-secret-syncer-bdkrt\" (UID: \"2693ad6a-f241-4165-8d2e-9aee83cc8ae8\") " pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:43.507615 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:43.507606 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs\") pod \"network-metrics-daemon-5nqdq\" (UID: \"bf7391f7-3b4a-4eb5-ab3a-b540543d4907\") " pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:43.507840 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:43.507757 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:47:43.507840 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:43.507818 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs podName:bf7391f7-3b4a-4eb5-ab3a-b540543d4907 nodeName:}" failed. No retries permitted until 2026-07-18 01:47:45.507799159 +0000 UTC m=+5.185223251 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs") pod "network-metrics-daemon-5nqdq" (UID: "bf7391f7-3b4a-4eb5-ab3a-b540543d4907") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:47:43.508291 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:43.508269 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:47:43.508374 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:43.508331 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret podName:2693ad6a-f241-4165-8d2e-9aee83cc8ae8 nodeName:}" failed. No retries permitted until 2026-07-18 01:47:45.508314442 +0000 UTC m=+5.185738534 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret") pod "global-pull-secret-syncer-bdkrt" (UID: "2693ad6a-f241-4165-8d2e-9aee83cc8ae8") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:47:43.608496 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:43.608460 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwchk\" (UniqueName: \"kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk\") pod \"network-check-target-2shnl\" (UID: \"af86de24-1529-4faa-9cdb-48c4cf6d6ede\") " pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:47:43.608738 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:43.608698 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:47:43.608738 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:43.608720 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:47:43.608738 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:43.608734 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qwchk for pod openshift-network-diagnostics/network-check-target-2shnl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:47:43.608902 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:43.608795 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk podName:af86de24-1529-4faa-9cdb-48c4cf6d6ede nodeName:}" failed. No retries permitted until 2026-07-18 01:47:45.608775118 +0000 UTC m=+5.286199213 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwchk" (UniqueName: "kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk") pod "network-check-target-2shnl" (UID: "af86de24-1529-4faa-9cdb-48c4cf6d6ede") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:47:43.829799 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:43.829685 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:42:41 +0000 UTC" deadline="2028-04-23 12:04:09.759832258 +0000 UTC" Jul 18 01:47:43.829799 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:43.829719 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15490h16m25.930117574s" Jul 18 01:47:43.889765 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:43.888906 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:43.889765 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:43.889048 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:47:43.889765 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:43.889452 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:47:43.889765 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:43.889540 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:47:43.889765 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:43.889597 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:43.889765 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:43.889684 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:47:44.200575 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:44.200496 2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:47:45.529784 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:45.528934 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret\") pod \"global-pull-secret-syncer-bdkrt\" (UID: \"2693ad6a-f241-4165-8d2e-9aee83cc8ae8\") " pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:45.529784 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:45.529004 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs\") pod \"network-metrics-daemon-5nqdq\" (UID: \"bf7391f7-3b4a-4eb5-ab3a-b540543d4907\") " pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:45.529784 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:45.529191 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:47:45.529784 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:45.529256 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs podName:bf7391f7-3b4a-4eb5-ab3a-b540543d4907 nodeName:}" failed. No retries permitted until 2026-07-18 01:47:49.529235445 +0000 UTC m=+9.206659536 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs") pod "network-metrics-daemon-5nqdq" (UID: "bf7391f7-3b4a-4eb5-ab3a-b540543d4907") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:47:45.529784 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:45.529674 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:47:45.529784 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:45.529723 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret podName:2693ad6a-f241-4165-8d2e-9aee83cc8ae8 nodeName:}" failed. No retries permitted until 2026-07-18 01:47:49.529708869 +0000 UTC m=+9.207132962 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret") pod "global-pull-secret-syncer-bdkrt" (UID: "2693ad6a-f241-4165-8d2e-9aee83cc8ae8") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:47:45.630726 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:45.629993 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwchk\" (UniqueName: \"kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk\") pod \"network-check-target-2shnl\" (UID: \"af86de24-1529-4faa-9cdb-48c4cf6d6ede\") " pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:47:45.630726 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:45.630174 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:47:45.630726 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:45.630197 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:47:45.630726 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:45.630212 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qwchk for pod openshift-network-diagnostics/network-check-target-2shnl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:47:45.630726 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:45.630268 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk podName:af86de24-1529-4faa-9cdb-48c4cf6d6ede nodeName:}" failed. No retries permitted until 2026-07-18 01:47:49.630251401 +0000 UTC m=+9.307675493 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwchk" (UniqueName: "kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk") pod "network-check-target-2shnl" (UID: "af86de24-1529-4faa-9cdb-48c4cf6d6ede") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:47:45.888542 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:45.888464 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:45.888717 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:45.888464 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:47:45.888717 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:45.888587 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:47:45.888717 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:45.888613 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:45.888867 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:45.888726 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:47:45.888867 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:45.888857 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:47:47.888774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:47.888715 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:47.888774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:47.888759 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:47:47.889417 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:47.888715 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:47.889417 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:47.888866 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:47:47.889417 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:47.889002 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:47:47.889417 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:47.889100 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:47:49.564073 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:49.563987 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret\") pod \"global-pull-secret-syncer-bdkrt\" (UID: \"2693ad6a-f241-4165-8d2e-9aee83cc8ae8\") " pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:49.564528 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:49.564073 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs\") pod \"network-metrics-daemon-5nqdq\" (UID: \"bf7391f7-3b4a-4eb5-ab3a-b540543d4907\") " pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:49.564528 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:49.564162 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:47:49.564528 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:49.564191 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:47:49.564528 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:49.564241 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret podName:2693ad6a-f241-4165-8d2e-9aee83cc8ae8 nodeName:}" failed. No retries permitted until 2026-07-18 01:47:57.56422001 +0000 UTC m=+17.241644104 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret") pod "global-pull-secret-syncer-bdkrt" (UID: "2693ad6a-f241-4165-8d2e-9aee83cc8ae8") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:47:49.564528 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:49.564261 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs podName:bf7391f7-3b4a-4eb5-ab3a-b540543d4907 nodeName:}" failed. No retries permitted until 2026-07-18 01:47:57.5642504 +0000 UTC m=+17.241674494 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs") pod "network-metrics-daemon-5nqdq" (UID: "bf7391f7-3b4a-4eb5-ab3a-b540543d4907") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:47:49.664760 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:49.664723 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwchk\" (UniqueName: \"kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk\") pod \"network-check-target-2shnl\" (UID: \"af86de24-1529-4faa-9cdb-48c4cf6d6ede\") " pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:47:49.664949 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:49.664925 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:47:49.664949 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:49.664947 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:47:49.665053 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:49.664959 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qwchk for pod openshift-network-diagnostics/network-check-target-2shnl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:47:49.665053 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:49.665016 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk podName:af86de24-1529-4faa-9cdb-48c4cf6d6ede nodeName:}" failed. No retries permitted until 2026-07-18 01:47:57.664998359 +0000 UTC m=+17.342422463 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwchk" (UniqueName: "kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk") pod "network-check-target-2shnl" (UID: "af86de24-1529-4faa-9cdb-48c4cf6d6ede") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:47:49.888752 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:49.888669 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:49.888917 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:49.888819 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:47:49.888917 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:49.888879 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:47:49.889033 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:49.888944 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:47:49.889033 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:49.888985 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:49.889118 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:49.889059 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:47:51.888041 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:51.887981 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:51.888524 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:51.888054 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:47:51.888524 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:51.888185 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:51.888524 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:51.888183 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:47:51.888524 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:51.888329 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:47:51.888524 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:51.888403 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:47:53.888902 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:53.888868 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:53.888902 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:53.888908 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:47:53.889322 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:53.888867 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:53.889322 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:53.889005 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:47:53.889322 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:53.889099 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:47:53.889322 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:53.889185 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:47:55.888680 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:55.888628 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:55.888680 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:55.888629 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:55.889137 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:55.888674 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:47:55.889137 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:55.888754 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:47:55.889137 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:55.888856 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:47:55.889137 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:55.888901 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:47:57.624202 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:57.624166 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret\") pod \"global-pull-secret-syncer-bdkrt\" (UID: \"2693ad6a-f241-4165-8d2e-9aee83cc8ae8\") " pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:57.624658 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:57.624228 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs\") pod \"network-metrics-daemon-5nqdq\" (UID: \"bf7391f7-3b4a-4eb5-ab3a-b540543d4907\") " pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:57.624658 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:57.624325 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:47:57.624658 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:57.624350 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:47:57.624658 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:57.624390 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret podName:2693ad6a-f241-4165-8d2e-9aee83cc8ae8 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:13.624374887 +0000 UTC m=+33.301798976 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret") pod "global-pull-secret-syncer-bdkrt" (UID: "2693ad6a-f241-4165-8d2e-9aee83cc8ae8") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:47:57.624658 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:57.624405 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs podName:bf7391f7-3b4a-4eb5-ab3a-b540543d4907 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:13.62439814 +0000 UTC m=+33.301822229 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs") pod "network-metrics-daemon-5nqdq" (UID: "bf7391f7-3b4a-4eb5-ab3a-b540543d4907") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:47:57.725194 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:57.725161 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwchk\" (UniqueName: \"kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk\") pod \"network-check-target-2shnl\" (UID: \"af86de24-1529-4faa-9cdb-48c4cf6d6ede\") " pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:47:57.725331 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:57.725301 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:47:57.725331 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:57.725318 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:47:57.725331 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:57.725327 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qwchk for pod openshift-network-diagnostics/network-check-target-2shnl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:47:57.725438 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:57.725379 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk podName:af86de24-1529-4faa-9cdb-48c4cf6d6ede nodeName:}" failed. No retries permitted until 2026-07-18 01:48:13.725364153 +0000 UTC m=+33.402788263 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwchk" (UniqueName: "kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk") pod "network-check-target-2shnl" (UID: "af86de24-1529-4faa-9cdb-48c4cf6d6ede") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:47:57.888786 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:57.888688 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:57.888786 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:57.888688 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:47:57.888993 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:57.888789 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:47:57.888993 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:57.888850 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:47:57.888993 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:57.888688 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:57.888993 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:57.888917 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:47:59.887999 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:59.887935 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:47:59.888465 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:59.888079 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:47:59.888465 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:47:59.888081 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:47:59.888465 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:59.888206 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:47:59.888465 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:59.888079 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:47:59.888465 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:47:59.888310 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:48:01.888886 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:01.888835 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:48:01.889362 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:01.888851 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:48:01.889362 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:01.888993 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:48:01.889362 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:01.888855 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:48:01.889362 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:01.889075 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:48:01.889362 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:01.889146 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:48:03.167462 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:03.167204 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xbkpg" event={"ID":"8baf11f1-2c27-4a7c-97c1-73f26856fe39","Type":"ContainerStarted","Data":"0ca09affbbe90a185c859becb559840f686a020e1dc1bdb134df2930701d53f6"} Jul 18 01:48:03.170825 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:03.170794 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 01:48:03.171223 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:03.171193 2570 generic.go:356] "Generic (PLEG): container finished" podID="51675a11-0cb3-4fa9-99db-3cd90c1355fa" containerID="2820a7184994868916b24719256375de2bf2acc7674c98a2b0c1c02ced2d5365" exitCode=1 Jul 18 01:48:03.171336 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:03.171316 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" event={"ID":"51675a11-0cb3-4fa9-99db-3cd90c1355fa","Type":"ContainerStarted","Data":"9c34b65bcde07475d0b81807ae0475494a316747c6b26baa61ee6e780449bcae"} Jul 18 01:48:03.171419 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:03.171401 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" event={"ID":"51675a11-0cb3-4fa9-99db-3cd90c1355fa","Type":"ContainerStarted","Data":"bafdd9d0c389f0f941a10d5720ade93d5b4dd820951f889d8946231023697385"} Jul 18 01:48:03.171483 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:03.171457 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" event={"ID":"51675a11-0cb3-4fa9-99db-3cd90c1355fa","Type":"ContainerStarted","Data":"1f4bcb021725ddc51eef14b7e4ac248324638560a34ed1adb3462ed33f373f78"} Jul 18 01:48:03.171539 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:03.171505 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" event={"ID":"51675a11-0cb3-4fa9-99db-3cd90c1355fa","Type":"ContainerStarted","Data":"6f6339a3da318babedc04584100eea8eeef122ad0914312893dae89c88afcbb7"} Jul 18 01:48:03.171598 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:03.171553 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" event={"ID":"51675a11-0cb3-4fa9-99db-3cd90c1355fa","Type":"ContainerDied","Data":"2820a7184994868916b24719256375de2bf2acc7674c98a2b0c1c02ced2d5365"} Jul 18 01:48:03.171678 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:03.171602 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" event={"ID":"51675a11-0cb3-4fa9-99db-3cd90c1355fa","Type":"ContainerStarted","Data":"2c4bd11d04cc6de3d8caa26fe44b55c6c2dca43c5d908dc0503f4706e66771d8"} Jul 18 01:48:03.173338 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:03.173312 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" event={"ID":"62155d42-0fc9-4747-a3dc-0f13a3c17f48","Type":"ContainerStarted","Data":"46cd592ef6482aa7bb520853f4dfbebbe3588a23bec7ff0b1a97e36b326c12d5"} Jul 18 01:48:03.181677 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:03.181426 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" event={"ID":"90ea7be93df65d66da652d9417af881e","Type":"ContainerStarted","Data":"ebf6656ad5d1e89028cb407b25f7bd54e96887ec9eddb40873461c0f04ce81a5"} Jul 18 01:48:03.185860 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:03.185310 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xbkpg" podStartSLOduration=2.232403361 podStartE2EDuration="22.185283312s" totalImagesPullingTime="19.952879951s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:03.18471918 +0000 UTC m=+22.862143292" watchObservedRunningTime="2026-07-18 01:48:03.185283312 +0000 UTC m=+22.862707424" Jul 18 01:48:03.197982 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:03.197938 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-151.ec2.internal" podStartSLOduration=22.197926149 podStartE2EDuration="22.197926149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:47:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:03.197776577 +0000 UTC m=+22.875200700" watchObservedRunningTime="2026-07-18 01:48:03.197926149 +0000 UTC m=+22.875350259" Jul 18 01:48:03.888496 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:03.888273 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:48:03.888757 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:03.888300 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:48:03.888757 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:03.888300 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:48:03.888757 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:03.888635 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:48:03.888757 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:03.888743 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:48:03.888941 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:03.888811 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:48:04.185309 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.185218 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wj9lj" event={"ID":"8af123e0-fca5-4d69-b7cd-dcfdc8e1750e","Type":"ContainerStarted","Data":"1ab5f9f1d37db590fa34f69a540cf2045d00a838b9e79d23e9bde53380d0a7b6"} Jul 18 01:48:04.186957 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.186925 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8xjzh" event={"ID":"270e5f34-05a1-4e34-828a-2ade34235111","Type":"ContainerStarted","Data":"9776c174a643bc07ff3ce3a557e6e5396fb71d0803cebb4831d7059b145c3935"} Jul 18 01:48:04.188747 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.188719 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-566ms" event={"ID":"abf94060-f8ec-42d6-8e79-2c6d3490ebf8","Type":"ContainerStarted","Data":"019afe3d32f4b5097c53b4693807f1305925265dbc59ca7fd18a51ff357d3d01"} Jul 18 01:48:04.190332 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.190309 2570 generic.go:356] "Generic (PLEG): container finished" podID="15fcf4b6dde762c7ce3ec0cc2a1506cd" containerID="4612a0573c27a03117ff755b411654522481096d665885904464db6186ad55eb" exitCode=0 Jul 18 01:48:04.190438 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.190392 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" event={"ID":"15fcf4b6dde762c7ce3ec0cc2a1506cd","Type":"ContainerDied","Data":"4612a0573c27a03117ff755b411654522481096d665885904464db6186ad55eb"} Jul 18 01:48:04.192292 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.192239 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-kj7rk" event={"ID":"05401e73-f450-4557-b68f-5092e860981c","Type":"ContainerStarted","Data":"4b9a47afe31ef22691407bdd40ae14046b042d344898d8340b9044f96f608f4f"} Jul 18 01:48:04.194041 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.193953 2570 generic.go:356] "Generic (PLEG): container finished" podID="d53a7743-60f0-4308-8165-6b27513efeb4" containerID="2eb58e333774d404334e0d35f1e9abbf6dbbc3e2634999a6b7549b9ca09dca4c" exitCode=0 Jul 18 01:48:04.194041 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.194025 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5lxc9" event={"ID":"d53a7743-60f0-4308-8165-6b27513efeb4","Type":"ContainerDied","Data":"2eb58e333774d404334e0d35f1e9abbf6dbbc3e2634999a6b7549b9ca09dca4c"} Jul 18 01:48:04.195714 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.195693 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" event={"ID":"d287d9c6-2390-44b3-902b-3cf24d7a32e0","Type":"ContainerStarted","Data":"2ddcce9450669dea8914c88807481cb22861fb732e78a61defd45cf994ae18b0"} Jul 18 01:48:04.201741 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.201696 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-w5hwx" podStartSLOduration=4.178256114 podStartE2EDuration="24.201681887s" totalImagesPullingTime="20.023425773s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:47:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:03.213740312 +0000 UTC m=+22.891164649" watchObservedRunningTime="2026-07-18 01:48:04.201681887 +0000 UTC m=+23.879105995" Jul 18 01:48:04.202485 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.202449 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wj9lj" podStartSLOduration=11.538870325 podStartE2EDuration="23.202434075s" totalImagesPullingTime="11.66356375s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:04.202114475 +0000 UTC m=+23.879538586" watchObservedRunningTime="2026-07-18 01:48:04.202434075 +0000 UTC m=+23.879858188" Jul 18 01:48:04.215024 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.214969 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-kj7rk" podStartSLOduration=3.30176505 podStartE2EDuration="23.214948139s" totalImagesPullingTime="19.913183089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:04.214383963 +0000 UTC m=+23.891808087" watchObservedRunningTime="2026-07-18 01:48:04.214948139 +0000 UTC m=+23.892372250" Jul 18 01:48:04.227901 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.227856 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8xjzh" podStartSLOduration=3.225902959 podStartE2EDuration="23.227841473s" totalImagesPullingTime="20.001938514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:04.227689527 +0000 UTC m=+23.905113638" watchObservedRunningTime="2026-07-18 01:48:04.227841473 +0000 UTC m=+23.905265583" Jul 18 01:48:04.275411 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.275362 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-566ms" podStartSLOduration=4.242448171 podStartE2EDuration="24.275346069s" totalImagesPullingTime="20.032897898s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:47:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:04.275042482 +0000 UTC m=+23.952466592" watchObservedRunningTime="2026-07-18 01:48:04.275346069 +0000 UTC m=+23.952770192" Jul 18 01:48:04.645934 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.645909 2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 18 01:48:04.850555 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.850365 2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-18T01:48:04.645929017Z","UUID":"4ac47b2d-bb01-424a-8fbb-6eef24be48f1","Handler":null,"Name":"","Endpoint":""} Jul 18 01:48:04.852946 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.852922 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 18 01:48:04.853071 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:04.852969 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 18 01:48:05.200485 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:05.200450 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" event={"ID":"d287d9c6-2390-44b3-902b-3cf24d7a32e0","Type":"ContainerStarted","Data":"312ff588c056e27dde6e36d8e0b001214d7c6b6b504ffda2fa8ef8d948fb7019"} Jul 18 01:48:05.202394 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:05.202356 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" event={"ID":"15fcf4b6dde762c7ce3ec0cc2a1506cd","Type":"ContainerStarted","Data":"b3cb46439c27d25a45a6140bcf49c3da5bababa23741472da4215e78f247e86a"} Jul 18 01:48:05.217809 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:05.217757 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-151.ec2.internal" podStartSLOduration=24.217736118 podStartE2EDuration="24.217736118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:47:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:05.217512747 +0000 UTC m=+24.894936859" watchObservedRunningTime="2026-07-18 01:48:05.217736118 +0000 UTC m=+24.895160231" Jul 18 01:48:05.888601 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:05.888560 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:48:05.888601 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:05.888581 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:48:05.888601 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:05.888594 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:48:05.889023 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:05.888731 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:48:05.889023 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:05.888810 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:48:05.889023 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:05.888902 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:48:06.208042 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:06.208010 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 01:48:06.208569 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:06.208503 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" event={"ID":"51675a11-0cb3-4fa9-99db-3cd90c1355fa","Type":"ContainerStarted","Data":"c839aa3a3626b78ab132c35026bf20bd3cd0b297817d1a826eec6918b3a8350e"} Jul 18 01:48:06.210722 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:06.210697 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" event={"ID":"d287d9c6-2390-44b3-902b-3cf24d7a32e0","Type":"ContainerStarted","Data":"565987cb5706806f90cfad4653d1b6f2f9b11052636c4c71ff7db919ece864b6"} Jul 18 01:48:06.225843 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:06.225766 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fkckl" podStartSLOduration=2.562457812 podStartE2EDuration="25.225752389s" totalImagesPullingTime="22.663294577s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:47:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:06.225670459 +0000 UTC m=+25.903094570" watchObservedRunningTime="2026-07-18 01:48:06.225752389 +0000 UTC m=+25.903176498" Jul 18 01:48:07.888657 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:07.888579 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:48:07.889423 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:07.888837 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:48:07.889423 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:07.888838 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:48:07.889423 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:07.888899 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:48:07.889423 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:07.889024 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:48:07.889423 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:07.889101 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:48:07.943476 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:07.943437 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wj9lj" Jul 18 01:48:07.946060 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:07.946026 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wj9lj" Jul 18 01:48:07.946667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:07.946651 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wj9lj" Jul 18 01:48:08.215934 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:08.215865 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wj9lj" Jul 18 01:48:09.221176 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:09.221011 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 01:48:09.221976 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:09.221523 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" event={"ID":"51675a11-0cb3-4fa9-99db-3cd90c1355fa","Type":"ContainerStarted","Data":"9c779292ae24eca2152554d7066517e09d5ec9286d8fd6bf308d7881613c3e98"} Jul 18 01:48:09.221976 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:09.221856 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:48:09.222110 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:09.222037 2570 scope.go:117] "RemoveContainer" containerID="2820a7184994868916b24719256375de2bf2acc7674c98a2b0c1c02ced2d5365" Jul 18 01:48:09.223487 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:09.223461 2570 generic.go:356] "Generic (PLEG): container finished" podID="d53a7743-60f0-4308-8165-6b27513efeb4" containerID="5f66bae927dead932c81ee8c2665cfcb3366a339d775ab6360906ad590ce7bb4" exitCode=0 Jul 18 01:48:09.223590 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:09.223547 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5lxc9" event={"ID":"d53a7743-60f0-4308-8165-6b27513efeb4","Type":"ContainerDied","Data":"5f66bae927dead932c81ee8c2665cfcb3366a339d775ab6360906ad590ce7bb4"} Jul 18 01:48:09.237909 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:09.237892 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:48:09.889140 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:09.889028 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:48:09.889140 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:09.889061 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:48:09.889446 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:09.889149 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:48:09.889446 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:09.889409 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:48:09.889446 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:09.889028 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:48:09.889602 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:09.889541 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:48:10.229192 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:10.229120 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 01:48:10.229555 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:10.229518 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" event={"ID":"51675a11-0cb3-4fa9-99db-3cd90c1355fa","Type":"ContainerStarted","Data":"4a3d8e2a07c9469627352fc90442e190b6220ab1607ea0b67680ebf64273af62"} Jul 18 01:48:10.229868 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:10.229793 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:48:10.229994 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:10.229915 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:48:10.244499 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:10.244473 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:48:10.256413 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:10.256371 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" podStartSLOduration=8.925750419 podStartE2EDuration="29.256357476s" totalImagesPullingTime="20.330607057s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:10.255168826 +0000 UTC m=+29.932593018" watchObservedRunningTime="2026-07-18 01:48:10.256357476 +0000 UTC m=+29.933781586" Jul 18 01:48:10.561281 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:10.561211 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2shnl"] Jul 18 01:48:10.561423 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:10.561313 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:48:10.561478 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:10.561420 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:48:10.563944 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:10.563915 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bdkrt"] Jul 18 01:48:10.564065 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:10.564018 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:48:10.564173 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:10.564150 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:48:10.564711 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:10.564679 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5nqdq"] Jul 18 01:48:10.564794 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:10.564774 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:48:10.564895 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:10.564863 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:48:11.233444 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:11.233405 2570 generic.go:356] "Generic (PLEG): container finished" podID="d53a7743-60f0-4308-8165-6b27513efeb4" containerID="9d6db669aeb78a7d6c1c85256b1f029b15518472faecaa19e1e72e8e1f209696" exitCode=0 Jul 18 01:48:11.233815 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:11.233479 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5lxc9" event={"ID":"d53a7743-60f0-4308-8165-6b27513efeb4","Type":"ContainerDied","Data":"9d6db669aeb78a7d6c1c85256b1f029b15518472faecaa19e1e72e8e1f209696"} Jul 18 01:48:11.888381 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:11.888346 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:48:11.888532 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:11.888396 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:48:11.888532 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:11.888470 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:48:11.888674 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:11.888588 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:48:12.888010 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:12.887985 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:48:12.888477 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:12.888087 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:48:13.239769 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:13.239690 2570 generic.go:356] "Generic (PLEG): container finished" podID="d53a7743-60f0-4308-8165-6b27513efeb4" containerID="789cea2ef497666495106a65f21edcf51ec6e9cecf2bf084164510c296e2a240" exitCode=0 Jul 18 01:48:13.239769 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:13.239730 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5lxc9" event={"ID":"d53a7743-60f0-4308-8165-6b27513efeb4","Type":"ContainerDied","Data":"789cea2ef497666495106a65f21edcf51ec6e9cecf2bf084164510c296e2a240"} Jul 18 01:48:13.651929 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:13.651849 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret\") pod \"global-pull-secret-syncer-bdkrt\" (UID: \"2693ad6a-f241-4165-8d2e-9aee83cc8ae8\") " pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:48:13.651929 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:13.651901 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs\") pod \"network-metrics-daemon-5nqdq\" (UID: \"bf7391f7-3b4a-4eb5-ab3a-b540543d4907\") " pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:48:13.652119 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:13.651994 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:13.652119 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:13.651996 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:13.652119 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:13.652045 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs podName:bf7391f7-3b4a-4eb5-ab3a-b540543d4907 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:45.652030768 +0000 UTC m=+65.329454856 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs") pod "network-metrics-daemon-5nqdq" (UID: "bf7391f7-3b4a-4eb5-ab3a-b540543d4907") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:13.652119 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:13.652074 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret podName:2693ad6a-f241-4165-8d2e-9aee83cc8ae8 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:45.65205123 +0000 UTC m=+65.329475318 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret") pod "global-pull-secret-syncer-bdkrt" (UID: "2693ad6a-f241-4165-8d2e-9aee83cc8ae8") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:13.752413 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:13.752378 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwchk\" (UniqueName: \"kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk\") pod \"network-check-target-2shnl\" (UID: \"af86de24-1529-4faa-9cdb-48c4cf6d6ede\") " pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:48:13.752592 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:13.752542 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:48:13.752592 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:13.752559 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:48:13.752592 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:13.752571 2570 projected.go:196] Error preparing data for projected volume kube-api-access-qwchk for pod openshift-network-diagnostics/network-check-target-2shnl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:13.752768 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:13.752628 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk podName:af86de24-1529-4faa-9cdb-48c4cf6d6ede nodeName:}" failed. No retries permitted until 2026-07-18 01:48:45.752612367 +0000 UTC m=+65.430036458 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-qwchk" (UniqueName: "kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk") pod "network-check-target-2shnl" (UID: "af86de24-1529-4faa-9cdb-48c4cf6d6ede") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:13.888969 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:13.888936 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:48:13.889341 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:13.888936 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:48:13.889341 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:13.889044 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2shnl" podUID="af86de24-1529-4faa-9cdb-48c4cf6d6ede" Jul 18 01:48:13.889341 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:13.889119 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bdkrt" podUID="2693ad6a-f241-4165-8d2e-9aee83cc8ae8" Jul 18 01:48:14.888406 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:14.888193 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:48:14.888602 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:14.888518 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5nqdq" podUID="bf7391f7-3b4a-4eb5-ab3a-b540543d4907" Jul 18 01:48:15.631127 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.631098 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-151.ec2.internal" event="NodeReady" Jul 18 01:48:15.631489 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.631232 2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 18 01:48:15.668287 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.668244 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5847499567-xn4cp"] Jul 18 01:48:15.691397 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.691372 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5847499567-xn4cp"] Jul 18 01:48:15.691551 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.691511 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.694277 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.694255 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 18 01:48:15.694277 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.694270 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 18 01:48:15.694479 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.694398 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-66smx\"" Jul 18 01:48:15.694845 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.694824 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 18 01:48:15.699654 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.699618 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 18 01:48:15.780219 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.780186 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bzrjc"] Jul 18 01:48:15.803711 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.803681 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:15.808458 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.808376 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 18 01:48:15.808695 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.808539 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ml5qw\"" Jul 18 01:48:15.809005 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.808986 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 18 01:48:15.809573 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.809532 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bzrjc"] Jul 18 01:48:15.869703 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.869670 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-bound-sa-token\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.869703 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.869700 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm8zc\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-kube-api-access-cm8zc\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.869921 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.869721 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/632a8dc2-b00b-419a-a1f7-416a4cfa380c-image-registry-private-configuration\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.869921 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.869739 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-certificates\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.869921 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.869760 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.869921 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.869774 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/632a8dc2-b00b-419a-a1f7-416a4cfa380c-ca-trust-extracted\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.869921 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.869856 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/632a8dc2-b00b-419a-a1f7-416a4cfa380c-installation-pull-secrets\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.869921 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.869909 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/632a8dc2-b00b-419a-a1f7-416a4cfa380c-trusted-ca\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.888853 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.888788 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:48:15.888961 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.888788 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:48:15.909925 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.909909 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 18 01:48:15.915084 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.915064 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vwvlx\"" Jul 18 01:48:15.915210 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.915126 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 18 01:48:15.915262 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.915229 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 01:48:15.916106 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.916080 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b78nn"] Jul 18 01:48:15.933417 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.933397 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b78nn" Jul 18 01:48:15.966964 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:15.966929 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"ingress-canary-b78nn\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress-canary\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="f9231edc-64b9-43d0-b939-e9043f91e610" pod="openshift-ingress-canary/ingress-canary-b78nn" Jul 18 01:48:15.968508 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:15.968485 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress-canary\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 18 01:48:15.968818 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:15.968798 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"canary-serving-cert\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress-canary\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" type="*v1.Secret" Jul 18 01:48:15.969045 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:15.969024 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-pf5pc\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress-canary\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pf5pc\"" type="*v1.Secret" Jul 18 01:48:15.970457 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.970442 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-bound-sa-token\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.970529 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.970465 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm8zc\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-kube-api-access-cm8zc\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.970529 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.970485 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/632a8dc2-b00b-419a-a1f7-416a4cfa380c-image-registry-private-configuration\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.970529 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.970503 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-certificates\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.970529 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.970522 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:15.970738 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.970628 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.970738 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.970660 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/632a8dc2-b00b-419a-a1f7-416a4cfa380c-ca-trust-extracted\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.970738 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.970687 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsbpn\" (UniqueName: \"kubernetes.io/projected/163a2e0f-86f1-4c36-ba43-6d097cbec349-kube-api-access-jsbpn\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:15.970738 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.970715 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/632a8dc2-b00b-419a-a1f7-416a4cfa380c-installation-pull-secrets\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.970905 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:15.970772 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:48:15.970905 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:15.970786 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5847499567-xn4cp: secret "image-registry-tls" not found Jul 18 01:48:15.970905 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.970790 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/163a2e0f-86f1-4c36-ba43-6d097cbec349-config-volume\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:15.970905 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.970820 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/632a8dc2-b00b-419a-a1f7-416a4cfa380c-trusted-ca\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.970905 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:15.970858 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls podName:632a8dc2-b00b-419a-a1f7-416a4cfa380c nodeName:}" failed. No retries permitted until 2026-07-18 01:48:16.4708377 +0000 UTC m=+36.148261816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls") pod "image-registry-5847499567-xn4cp" (UID: "632a8dc2-b00b-419a-a1f7-416a4cfa380c") : secret "image-registry-tls" not found Jul 18 01:48:15.971094 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.970943 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163a2e0f-86f1-4c36-ba43-6d097cbec349-tmp-dir\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:15.980193 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.980176 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 18 01:48:15.980584 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.980564 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/632a8dc2-b00b-419a-a1f7-416a4cfa380c-ca-trust-extracted\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.980831 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.980808 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-certificates\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.980945 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.980928 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/632a8dc2-b00b-419a-a1f7-416a4cfa380c-installation-pull-secrets\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.981015 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.980945 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/632a8dc2-b00b-419a-a1f7-416a4cfa380c-image-registry-private-configuration\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.981091 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.981069 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/632a8dc2-b00b-419a-a1f7-416a4cfa380c-trusted-ca\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:15.997810 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:15.997783 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-bound-sa-token\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:16.025233 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.025196 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b78nn"] Jul 18 01:48:16.032070 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.032040 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm8zc\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-kube-api-access-cm8zc\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:16.072064 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.072027 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsbpn\" (UniqueName: \"kubernetes.io/projected/163a2e0f-86f1-4c36-ba43-6d097cbec349-kube-api-access-jsbpn\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:16.072248 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.072087 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f9231edc-64b9-43d0-b939-e9043f91e610-cert\") pod \"ingress-canary-b78nn\" (UID: \"f9231edc-64b9-43d0-b939-e9043f91e610\") " pod="openshift-ingress-canary/ingress-canary-b78nn" Jul 18 01:48:16.072248 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.072113 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/163a2e0f-86f1-4c36-ba43-6d097cbec349-config-volume\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:16.072248 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.072135 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163a2e0f-86f1-4c36-ba43-6d097cbec349-tmp-dir\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:16.072248 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.072165 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh56m\" (UniqueName: \"kubernetes.io/projected/f9231edc-64b9-43d0-b939-e9043f91e610-kube-api-access-kh56m\") pod \"ingress-canary-b78nn\" (UID: \"f9231edc-64b9-43d0-b939-e9043f91e610\") " pod="openshift-ingress-canary/ingress-canary-b78nn" Jul 18 01:48:16.072452 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.072365 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:16.072516 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:16.072499 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:48:16.072561 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.072534 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/163a2e0f-86f1-4c36-ba43-6d097cbec349-tmp-dir\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:16.072599 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:16.072570 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls podName:163a2e0f-86f1-4c36-ba43-6d097cbec349 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:16.57255266 +0000 UTC m=+36.249976749 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls") pod "dns-default-bzrjc" (UID: "163a2e0f-86f1-4c36-ba43-6d097cbec349") : secret "dns-default-metrics-tls" not found Jul 18 01:48:16.072864 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.072845 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/163a2e0f-86f1-4c36-ba43-6d097cbec349-config-volume\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:16.094122 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.094090 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsbpn\" (UniqueName: \"kubernetes.io/projected/163a2e0f-86f1-4c36-ba43-6d097cbec349-kube-api-access-jsbpn\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:16.173358 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.173265 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f9231edc-64b9-43d0-b939-e9043f91e610-cert\") pod \"ingress-canary-b78nn\" (UID: \"f9231edc-64b9-43d0-b939-e9043f91e610\") " pod="openshift-ingress-canary/ingress-canary-b78nn" Jul 18 01:48:16.173358 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.173315 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh56m\" (UniqueName: \"kubernetes.io/projected/f9231edc-64b9-43d0-b939-e9043f91e610-kube-api-access-kh56m\") pod \"ingress-canary-b78nn\" (UID: \"f9231edc-64b9-43d0-b939-e9043f91e610\") " pod="openshift-ingress-canary/ingress-canary-b78nn" Jul 18 01:48:16.476739 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.476664 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:16.476881 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:16.476847 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:48:16.476881 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:16.476870 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5847499567-xn4cp: secret "image-registry-tls" not found Jul 18 01:48:16.476954 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:16.476940 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls podName:632a8dc2-b00b-419a-a1f7-416a4cfa380c nodeName:}" failed. No retries permitted until 2026-07-18 01:48:17.476920075 +0000 UTC m=+37.154344181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls") pod "image-registry-5847499567-xn4cp" (UID: "632a8dc2-b00b-419a-a1f7-416a4cfa380c") : secret "image-registry-tls" not found Jul 18 01:48:16.577474 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.577431 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:16.577664 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:16.577569 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:48:16.577664 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:16.577633 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls podName:163a2e0f-86f1-4c36-ba43-6d097cbec349 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:17.577617608 +0000 UTC m=+37.255041699 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls") pod "dns-default-bzrjc" (UID: "163a2e0f-86f1-4c36-ba43-6d097cbec349") : secret "dns-default-metrics-tls" not found Jul 18 01:48:16.888134 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.888095 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:48:16.890940 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.890911 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tqgxz\"" Jul 18 01:48:16.890940 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:16.890912 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 01:48:17.173793 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:17.173713 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: failed to sync secret cache: timed out waiting for the condition Jul 18 01:48:17.173959 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:17.173801 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9231edc-64b9-43d0-b939-e9043f91e610-cert podName:f9231edc-64b9-43d0-b939-e9043f91e610 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:17.673779283 +0000 UTC m=+37.351203377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f9231edc-64b9-43d0-b939-e9043f91e610-cert") pod "ingress-canary-b78nn" (UID: "f9231edc-64b9-43d0-b939-e9043f91e610") : failed to sync secret cache: timed out waiting for the condition Jul 18 01:48:17.188340 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:17.188303 2570 projected.go:291] Couldn't get configMap openshift-ingress-canary/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jul 18 01:48:17.188340 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:17.188342 2570 projected.go:196] Error preparing data for projected volume kube-api-access-kh56m for pod openshift-ingress-canary/ingress-canary-b78nn: failed to sync configmap cache: timed out waiting for the condition Jul 18 01:48:17.188534 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:17.188396 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f9231edc-64b9-43d0-b939-e9043f91e610-kube-api-access-kh56m podName:f9231edc-64b9-43d0-b939-e9043f91e610 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:17.688380433 +0000 UTC m=+37.365804523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-kh56m" (UniqueName: "kubernetes.io/projected/f9231edc-64b9-43d0-b939-e9043f91e610-kube-api-access-kh56m") pod "ingress-canary-b78nn" (UID: "f9231edc-64b9-43d0-b939-e9043f91e610") : failed to sync configmap cache: timed out waiting for the condition Jul 18 01:48:17.212075 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:17.212046 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 18 01:48:17.391356 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:17.391325 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pf5pc\"" Jul 18 01:48:17.418764 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:17.418732 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 18 01:48:17.486541 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:17.486447 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:17.486742 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:17.486571 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:48:17.486742 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:17.486588 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5847499567-xn4cp: secret "image-registry-tls" not found Jul 18 01:48:17.486742 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:17.486685 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls podName:632a8dc2-b00b-419a-a1f7-416a4cfa380c nodeName:}" failed. No retries permitted until 2026-07-18 01:48:19.486665113 +0000 UTC m=+39.164089222 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls") pod "image-registry-5847499567-xn4cp" (UID: "632a8dc2-b00b-419a-a1f7-416a4cfa380c") : secret "image-registry-tls" not found Jul 18 01:48:17.587731 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:17.587697 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:17.587868 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:17.587839 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:48:17.587912 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:17.587902 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls podName:163a2e0f-86f1-4c36-ba43-6d097cbec349 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:19.587884411 +0000 UTC m=+39.265308517 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls") pod "dns-default-bzrjc" (UID: "163a2e0f-86f1-4c36-ba43-6d097cbec349") : secret "dns-default-metrics-tls" not found Jul 18 01:48:17.688702 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:17.688666 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f9231edc-64b9-43d0-b939-e9043f91e610-cert\") pod \"ingress-canary-b78nn\" (UID: \"f9231edc-64b9-43d0-b939-e9043f91e610\") " pod="openshift-ingress-canary/ingress-canary-b78nn" Jul 18 01:48:17.688840 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:17.688711 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh56m\" (UniqueName: \"kubernetes.io/projected/f9231edc-64b9-43d0-b939-e9043f91e610-kube-api-access-kh56m\") pod \"ingress-canary-b78nn\" (UID: \"f9231edc-64b9-43d0-b939-e9043f91e610\") " pod="openshift-ingress-canary/ingress-canary-b78nn" Jul 18 01:48:17.691264 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:17.691231 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f9231edc-64b9-43d0-b939-e9043f91e610-cert\") pod \"ingress-canary-b78nn\" (UID: \"f9231edc-64b9-43d0-b939-e9043f91e610\") " pod="openshift-ingress-canary/ingress-canary-b78nn" Jul 18 01:48:17.691366 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:17.691313 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh56m\" (UniqueName: \"kubernetes.io/projected/f9231edc-64b9-43d0-b939-e9043f91e610-kube-api-access-kh56m\") pod \"ingress-canary-b78nn\" (UID: \"f9231edc-64b9-43d0-b939-e9043f91e610\") " pod="openshift-ingress-canary/ingress-canary-b78nn" Jul 18 01:48:17.741659 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:17.741559 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b78nn" Jul 18 01:48:18.116206 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:18.115920 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b78nn"] Jul 18 01:48:18.122003 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:48:18.121981 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9231edc_64b9_43d0_b939_e9043f91e610.slice/crio-7e2f30aec6f6a0a39ec170c0d6a02f31e15a7b27100ae707154b1314d24fec67 WatchSource:0}: Error finding container 7e2f30aec6f6a0a39ec170c0d6a02f31e15a7b27100ae707154b1314d24fec67: Status 404 returned error can't find the container with id 7e2f30aec6f6a0a39ec170c0d6a02f31e15a7b27100ae707154b1314d24fec67 Jul 18 01:48:18.253027 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:18.252992 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b78nn" event={"ID":"f9231edc-64b9-43d0-b939-e9043f91e610","Type":"ContainerStarted","Data":"7e2f30aec6f6a0a39ec170c0d6a02f31e15a7b27100ae707154b1314d24fec67"} Jul 18 01:48:18.255558 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:18.255532 2570 generic.go:356] "Generic (PLEG): container finished" podID="d53a7743-60f0-4308-8165-6b27513efeb4" containerID="62e162269dfbdb7529131978928f268df65817fcae23f0f0c2f400855821c086" exitCode=0 Jul 18 01:48:18.255687 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:18.255585 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5lxc9" event={"ID":"d53a7743-60f0-4308-8165-6b27513efeb4","Type":"ContainerDied","Data":"62e162269dfbdb7529131978928f268df65817fcae23f0f0c2f400855821c086"} Jul 18 01:48:19.262106 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:19.262053 2570 generic.go:356] "Generic (PLEG): container finished" podID="d53a7743-60f0-4308-8165-6b27513efeb4" containerID="b70ccbe57ebc7974dc802d793384a8b4fb6e5ddb69359297bc167c81cced50b0" exitCode=0 Jul 18 01:48:19.262106 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:19.262102 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5lxc9" event={"ID":"d53a7743-60f0-4308-8165-6b27513efeb4","Type":"ContainerDied","Data":"b70ccbe57ebc7974dc802d793384a8b4fb6e5ddb69359297bc167c81cced50b0"} Jul 18 01:48:19.502335 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:19.502299 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:19.502507 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:19.502480 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:48:19.502507 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:19.502504 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5847499567-xn4cp: secret "image-registry-tls" not found Jul 18 01:48:19.502604 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:19.502577 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls podName:632a8dc2-b00b-419a-a1f7-416a4cfa380c nodeName:}" failed. No retries permitted until 2026-07-18 01:48:23.502555646 +0000 UTC m=+43.179979752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls") pod "image-registry-5847499567-xn4cp" (UID: "632a8dc2-b00b-419a-a1f7-416a4cfa380c") : secret "image-registry-tls" not found Jul 18 01:48:19.602985 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:19.602953 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:19.603153 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:19.603107 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:48:19.603203 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:19.603179 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls podName:163a2e0f-86f1-4c36-ba43-6d097cbec349 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:23.603158765 +0000 UTC m=+43.280582853 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls") pod "dns-default-bzrjc" (UID: "163a2e0f-86f1-4c36-ba43-6d097cbec349") : secret "dns-default-metrics-tls" not found Jul 18 01:48:20.269027 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:20.268993 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5lxc9" event={"ID":"d53a7743-60f0-4308-8165-6b27513efeb4","Type":"ContainerStarted","Data":"3fedd22f8f33d3a3d4a2a3ae5fb53cf683af2eaf7f2cd1cf13e0e77c9dcf0f2d"} Jul 18 01:48:20.291426 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:20.291367 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5lxc9" podStartSLOduration=7.962601076 podStartE2EDuration="39.291346997s" totalImagesPullingTime="31.328745921s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:47:41 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:20.290078716 +0000 UTC m=+39.967502838" watchObservedRunningTime="2026-07-18 01:48:20.291346997 +0000 UTC m=+39.968771108" Jul 18 01:48:21.273461 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:21.273146 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b78nn" event={"ID":"f9231edc-64b9-43d0-b939-e9043f91e610","Type":"ContainerStarted","Data":"051adeb9bd39462ddc033931bf5ea3b8b789978f904163dd0f41a541882197e3"} Jul 18 01:48:21.288403 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:21.288357 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b78nn" podStartSLOduration=3.5778565220000003 podStartE2EDuration="6.288342837s" totalImagesPullingTime="2.710486315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:21.287213547 +0000 UTC m=+40.964637657" watchObservedRunningTime="2026-07-18 01:48:21.288342837 +0000 UTC m=+40.965767002" Jul 18 01:48:23.533485 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:23.533441 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:23.533895 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:23.533599 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:48:23.533895 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:23.533621 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5847499567-xn4cp: secret "image-registry-tls" not found Jul 18 01:48:23.533895 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:23.533698 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls podName:632a8dc2-b00b-419a-a1f7-416a4cfa380c nodeName:}" failed. No retries permitted until 2026-07-18 01:48:31.533683107 +0000 UTC m=+51.211107196 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls") pod "image-registry-5847499567-xn4cp" (UID: "632a8dc2-b00b-419a-a1f7-416a4cfa380c") : secret "image-registry-tls" not found Jul 18 01:48:23.634410 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:23.634368 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:23.634565 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:23.634531 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:48:23.634609 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:23.634594 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls podName:163a2e0f-86f1-4c36-ba43-6d097cbec349 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:31.634578547 +0000 UTC m=+51.312002635 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls") pod "dns-default-bzrjc" (UID: "163a2e0f-86f1-4c36-ba43-6d097cbec349") : secret "dns-default-metrics-tls" not found Jul 18 01:48:31.595712 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:31.595672 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:31.596197 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:31.595823 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:48:31.596197 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:31.595841 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5847499567-xn4cp: secret "image-registry-tls" not found Jul 18 01:48:31.596197 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:31.595906 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls podName:632a8dc2-b00b-419a-a1f7-416a4cfa380c nodeName:}" failed. No retries permitted until 2026-07-18 01:48:47.59589047 +0000 UTC m=+67.273314559 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls") pod "image-registry-5847499567-xn4cp" (UID: "632a8dc2-b00b-419a-a1f7-416a4cfa380c") : secret "image-registry-tls" not found Jul 18 01:48:31.696689 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:31.696631 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:31.696850 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:31.696781 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:48:31.696850 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:31.696841 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls podName:163a2e0f-86f1-4c36-ba43-6d097cbec349 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:47.696826101 +0000 UTC m=+67.374250190 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls") pod "dns-default-bzrjc" (UID: "163a2e0f-86f1-4c36-ba43-6d097cbec349") : secret "dns-default-metrics-tls" not found Jul 18 01:48:35.867908 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.867867 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-8m75k"] Jul 18 01:48:35.870398 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.870382 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn"] Jul 18 01:48:35.870515 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.870498 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:48:35.872923 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.872905 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn" Jul 18 01:48:35.874266 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.874248 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 18 01:48:35.874341 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.874300 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 18 01:48:35.874466 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.874452 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 18 01:48:35.875514 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.875496 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-bdh7n\"" Jul 18 01:48:35.875722 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.875707 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 18 01:48:35.875806 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.875728 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 18 01:48:35.875806 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.875738 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 18 01:48:35.875806 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.875753 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 18 01:48:35.876052 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.875828 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 18 01:48:35.876052 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.875918 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-lr6xc\"" Jul 18 01:48:35.883106 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.883086 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 18 01:48:35.886939 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.886919 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-8m75k"] Jul 18 01:48:35.887539 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.887523 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn"] Jul 18 01:48:35.930787 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.930754 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eecae00a-6435-4154-9386-988f7365b0cb-config\") pod \"service-ca-operator-5bcbc94c78-czfpn\" (UID: \"eecae00a-6435-4154-9386-988f7365b0cb\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn" Jul 18 01:48:35.930914 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.930797 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x5md\" (UniqueName: \"kubernetes.io/projected/eecae00a-6435-4154-9386-988f7365b0cb-kube-api-access-6x5md\") pod \"service-ca-operator-5bcbc94c78-czfpn\" (UID: \"eecae00a-6435-4154-9386-988f7365b0cb\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn" Jul 18 01:48:35.930914 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.930841 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d5439579-cb04-47d9-99a6-60c482c4afbd-trusted-ca\") pod \"console-operator-565b7b569d-8m75k\" (UID: \"d5439579-cb04-47d9-99a6-60c482c4afbd\") " pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:48:35.930914 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.930881 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eecae00a-6435-4154-9386-988f7365b0cb-serving-cert\") pod \"service-ca-operator-5bcbc94c78-czfpn\" (UID: \"eecae00a-6435-4154-9386-988f7365b0cb\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn" Jul 18 01:48:35.931027 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.930933 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d5439579-cb04-47d9-99a6-60c482c4afbd-serving-cert\") pod \"console-operator-565b7b569d-8m75k\" (UID: \"d5439579-cb04-47d9-99a6-60c482c4afbd\") " pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:48:35.931027 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.931005 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d5439579-cb04-47d9-99a6-60c482c4afbd-config\") pod \"console-operator-565b7b569d-8m75k\" (UID: \"d5439579-cb04-47d9-99a6-60c482c4afbd\") " pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:48:35.931102 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:35.931035 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xswl2\" (UniqueName: \"kubernetes.io/projected/d5439579-cb04-47d9-99a6-60c482c4afbd-kube-api-access-xswl2\") pod \"console-operator-565b7b569d-8m75k\" (UID: \"d5439579-cb04-47d9-99a6-60c482c4afbd\") " pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:48:36.032123 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.032088 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eecae00a-6435-4154-9386-988f7365b0cb-config\") pod \"service-ca-operator-5bcbc94c78-czfpn\" (UID: \"eecae00a-6435-4154-9386-988f7365b0cb\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn" Jul 18 01:48:36.032123 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.032125 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x5md\" (UniqueName: \"kubernetes.io/projected/eecae00a-6435-4154-9386-988f7365b0cb-kube-api-access-6x5md\") pod \"service-ca-operator-5bcbc94c78-czfpn\" (UID: \"eecae00a-6435-4154-9386-988f7365b0cb\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn" Jul 18 01:48:36.032385 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.032150 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d5439579-cb04-47d9-99a6-60c482c4afbd-trusted-ca\") pod \"console-operator-565b7b569d-8m75k\" (UID: \"d5439579-cb04-47d9-99a6-60c482c4afbd\") " pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:48:36.032385 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.032302 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eecae00a-6435-4154-9386-988f7365b0cb-serving-cert\") pod \"service-ca-operator-5bcbc94c78-czfpn\" (UID: \"eecae00a-6435-4154-9386-988f7365b0cb\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn" Jul 18 01:48:36.032385 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.032335 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d5439579-cb04-47d9-99a6-60c482c4afbd-serving-cert\") pod \"console-operator-565b7b569d-8m75k\" (UID: \"d5439579-cb04-47d9-99a6-60c482c4afbd\") " pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:48:36.032385 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.032384 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d5439579-cb04-47d9-99a6-60c482c4afbd-config\") pod \"console-operator-565b7b569d-8m75k\" (UID: \"d5439579-cb04-47d9-99a6-60c482c4afbd\") " pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:48:36.032591 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.032416 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xswl2\" (UniqueName: \"kubernetes.io/projected/d5439579-cb04-47d9-99a6-60c482c4afbd-kube-api-access-xswl2\") pod \"console-operator-565b7b569d-8m75k\" (UID: \"d5439579-cb04-47d9-99a6-60c482c4afbd\") " pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:48:36.033028 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.033004 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d5439579-cb04-47d9-99a6-60c482c4afbd-trusted-ca\") pod \"console-operator-565b7b569d-8m75k\" (UID: \"d5439579-cb04-47d9-99a6-60c482c4afbd\") " pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:48:36.033028 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.033018 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d5439579-cb04-47d9-99a6-60c482c4afbd-config\") pod \"console-operator-565b7b569d-8m75k\" (UID: \"d5439579-cb04-47d9-99a6-60c482c4afbd\") " pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:48:36.033186 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.033004 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eecae00a-6435-4154-9386-988f7365b0cb-config\") pod \"service-ca-operator-5bcbc94c78-czfpn\" (UID: \"eecae00a-6435-4154-9386-988f7365b0cb\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn" Jul 18 01:48:36.035955 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.035926 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eecae00a-6435-4154-9386-988f7365b0cb-serving-cert\") pod \"service-ca-operator-5bcbc94c78-czfpn\" (UID: \"eecae00a-6435-4154-9386-988f7365b0cb\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn" Jul 18 01:48:36.036113 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.036094 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d5439579-cb04-47d9-99a6-60c482c4afbd-serving-cert\") pod \"console-operator-565b7b569d-8m75k\" (UID: \"d5439579-cb04-47d9-99a6-60c482c4afbd\") " pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:48:36.039291 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.039265 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x5md\" (UniqueName: \"kubernetes.io/projected/eecae00a-6435-4154-9386-988f7365b0cb-kube-api-access-6x5md\") pod \"service-ca-operator-5bcbc94c78-czfpn\" (UID: \"eecae00a-6435-4154-9386-988f7365b0cb\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn" Jul 18 01:48:36.039587 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.039570 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xswl2\" (UniqueName: \"kubernetes.io/projected/d5439579-cb04-47d9-99a6-60c482c4afbd-kube-api-access-xswl2\") pod \"console-operator-565b7b569d-8m75k\" (UID: \"d5439579-cb04-47d9-99a6-60c482c4afbd\") " pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:48:36.056091 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.056070 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8"] Jul 18 01:48:36.059904 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.059888 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" Jul 18 01:48:36.061996 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.061973 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 18 01:48:36.062233 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.062212 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-s7k8d\"" Jul 18 01:48:36.062336 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.062237 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 18 01:48:36.062336 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.062253 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 18 01:48:36.063036 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.063016 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 18 01:48:36.069485 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.069462 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8"] Jul 18 01:48:36.132839 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.132760 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e8e164cc-e6b4-4ebd-b122-e383719192d0-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-rh4q8\" (UID: \"e8e164cc-e6b4-4ebd-b122-e383719192d0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" Jul 18 01:48:36.132839 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.132807 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-rh4q8\" (UID: \"e8e164cc-e6b4-4ebd-b122-e383719192d0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" Jul 18 01:48:36.132839 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.132828 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnjqn\" (UniqueName: \"kubernetes.io/projected/e8e164cc-e6b4-4ebd-b122-e383719192d0-kube-api-access-wnjqn\") pod \"cluster-monitoring-operator-76cf8966f8-rh4q8\" (UID: \"e8e164cc-e6b4-4ebd-b122-e383719192d0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" Jul 18 01:48:36.180796 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.180766 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:48:36.185476 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.185460 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn" Jul 18 01:48:36.234744 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.234713 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e8e164cc-e6b4-4ebd-b122-e383719192d0-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-rh4q8\" (UID: \"e8e164cc-e6b4-4ebd-b122-e383719192d0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" Jul 18 01:48:36.234878 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.234781 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-rh4q8\" (UID: \"e8e164cc-e6b4-4ebd-b122-e383719192d0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" Jul 18 01:48:36.234878 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.234810 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnjqn\" (UniqueName: \"kubernetes.io/projected/e8e164cc-e6b4-4ebd-b122-e383719192d0-kube-api-access-wnjqn\") pod \"cluster-monitoring-operator-76cf8966f8-rh4q8\" (UID: \"e8e164cc-e6b4-4ebd-b122-e383719192d0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" Jul 18 01:48:36.235883 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.235857 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e8e164cc-e6b4-4ebd-b122-e383719192d0-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-rh4q8\" (UID: \"e8e164cc-e6b4-4ebd-b122-e383719192d0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" Jul 18 01:48:36.235991 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:36.235971 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:36.236047 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:36.236026 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls podName:e8e164cc-e6b4-4ebd-b122-e383719192d0 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:36.736009175 +0000 UTC m=+56.413433267 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-rh4q8" (UID: "e8e164cc-e6b4-4ebd-b122-e383719192d0") : secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:36.256663 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.249582 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnjqn\" (UniqueName: \"kubernetes.io/projected/e8e164cc-e6b4-4ebd-b122-e383719192d0-kube-api-access-wnjqn\") pod \"cluster-monitoring-operator-76cf8966f8-rh4q8\" (UID: \"e8e164cc-e6b4-4ebd-b122-e383719192d0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" Jul 18 01:48:36.325814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.325782 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-8m75k"] Jul 18 01:48:36.329211 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:48:36.329185 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5439579_cb04_47d9_99a6_60c482c4afbd.slice/crio-0fe5da3614f76205cba234416b89492eeb117fd95e00f695055b26b2d2b0b988 WatchSource:0}: Error finding container 0fe5da3614f76205cba234416b89492eeb117fd95e00f695055b26b2d2b0b988: Status 404 returned error can't find the container with id 0fe5da3614f76205cba234416b89492eeb117fd95e00f695055b26b2d2b0b988 Jul 18 01:48:36.341206 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.341179 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn"] Jul 18 01:48:36.344230 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:48:36.344198 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeecae00a_6435_4154_9386_988f7365b0cb.slice/crio-a4082ba8e8ff7c5dd7b6d79ce5073f1579f83dfd258d2fe0d62588aa94e882a6 WatchSource:0}: Error finding container a4082ba8e8ff7c5dd7b6d79ce5073f1579f83dfd258d2fe0d62588aa94e882a6: Status 404 returned error can't find the container with id a4082ba8e8ff7c5dd7b6d79ce5073f1579f83dfd258d2fe0d62588aa94e882a6 Jul 18 01:48:36.738992 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:36.738951 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-rh4q8\" (UID: \"e8e164cc-e6b4-4ebd-b122-e383719192d0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" Jul 18 01:48:36.739176 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:36.739067 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:36.739176 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:36.739131 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls podName:e8e164cc-e6b4-4ebd-b122-e383719192d0 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:37.739116698 +0000 UTC m=+57.416540786 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-rh4q8" (UID: "e8e164cc-e6b4-4ebd-b122-e383719192d0") : secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:37.312288 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:37.312240 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn" event={"ID":"eecae00a-6435-4154-9386-988f7365b0cb","Type":"ContainerStarted","Data":"a4082ba8e8ff7c5dd7b6d79ce5073f1579f83dfd258d2fe0d62588aa94e882a6"} Jul 18 01:48:37.313591 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:37.313527 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-8m75k" event={"ID":"d5439579-cb04-47d9-99a6-60c482c4afbd","Type":"ContainerStarted","Data":"0fe5da3614f76205cba234416b89492eeb117fd95e00f695055b26b2d2b0b988"} Jul 18 01:48:37.748436 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:37.748343 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-rh4q8\" (UID: \"e8e164cc-e6b4-4ebd-b122-e383719192d0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" Jul 18 01:48:37.748604 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:37.748521 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:37.748693 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:37.748605 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls podName:e8e164cc-e6b4-4ebd-b122-e383719192d0 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:39.74858233 +0000 UTC m=+59.426006458 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-rh4q8" (UID: "e8e164cc-e6b4-4ebd-b122-e383719192d0") : secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:39.320730 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.320694 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn" event={"ID":"eecae00a-6435-4154-9386-988f7365b0cb","Type":"ContainerStarted","Data":"8f77beadf161857885f6701369760e27e39a457b67c4d19f4f20623260efc63c"} Jul 18 01:48:39.322320 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.322300 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/0.log" Jul 18 01:48:39.322444 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.322334 2570 generic.go:356] "Generic (PLEG): container finished" podID="d5439579-cb04-47d9-99a6-60c482c4afbd" containerID="5cb762874f6041806f2b89d6d2c7df4873a6859d6a40c48a602c55d5680a6c4d" exitCode=255 Jul 18 01:48:39.322444 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.322364 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-8m75k" event={"ID":"d5439579-cb04-47d9-99a6-60c482c4afbd","Type":"ContainerDied","Data":"5cb762874f6041806f2b89d6d2c7df4873a6859d6a40c48a602c55d5680a6c4d"} Jul 18 01:48:39.322624 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.322609 2570 scope.go:117] "RemoveContainer" containerID="5cb762874f6041806f2b89d6d2c7df4873a6859d6a40c48a602c55d5680a6c4d" Jul 18 01:48:39.336866 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.336829 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn" podStartSLOduration=1.860153285 podStartE2EDuration="4.336814166s" totalImagesPullingTime="2.476660881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:39.336064967 +0000 UTC m=+59.013489079" watchObservedRunningTime="2026-07-18 01:48:39.336814166 +0000 UTC m=+59.014238282" Jul 18 01:48:39.623192 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.623109 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4"] Jul 18 01:48:39.625923 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.625908 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4" Jul 18 01:48:39.628180 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.628147 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 18 01:48:39.628333 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.628301 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 18 01:48:39.628446 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.628354 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-g72gf\"" Jul 18 01:48:39.635957 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.635939 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4"] Jul 18 01:48:39.666416 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.666385 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f2d2e933-102a-44e0-9813-093c6ff90178-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-b76q4\" (UID: \"f2d2e933-102a-44e0-9813-093c6ff90178\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4" Jul 18 01:48:39.666599 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.666495 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-b76q4\" (UID: \"f2d2e933-102a-44e0-9813-093c6ff90178\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4" Jul 18 01:48:39.767239 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.767205 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f2d2e933-102a-44e0-9813-093c6ff90178-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-b76q4\" (UID: \"f2d2e933-102a-44e0-9813-093c6ff90178\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4" Jul 18 01:48:39.767435 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.767292 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-rh4q8\" (UID: \"e8e164cc-e6b4-4ebd-b122-e383719192d0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" Jul 18 01:48:39.767435 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:39.767389 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:39.767435 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.767415 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-b76q4\" (UID: \"f2d2e933-102a-44e0-9813-093c6ff90178\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4" Jul 18 01:48:39.767612 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:39.767456 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls podName:e8e164cc-e6b4-4ebd-b122-e383719192d0 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:43.767437073 +0000 UTC m=+63.444861162 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-rh4q8" (UID: "e8e164cc-e6b4-4ebd-b122-e383719192d0") : secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:39.767612 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:39.767514 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 18 01:48:39.767612 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:39.767564 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert podName:f2d2e933-102a-44e0-9813-093c6ff90178 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:40.267548094 +0000 UTC m=+59.944972200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-b76q4" (UID: "f2d2e933-102a-44e0-9813-093c6ff90178") : secret "networking-console-plugin-cert" not found Jul 18 01:48:39.767863 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:39.767844 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f2d2e933-102a-44e0-9813-093c6ff90178-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-b76q4\" (UID: \"f2d2e933-102a-44e0-9813-093c6ff90178\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4" Jul 18 01:48:40.271902 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:40.271866 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-b76q4\" (UID: \"f2d2e933-102a-44e0-9813-093c6ff90178\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4" Jul 18 01:48:40.272067 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:40.272017 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 18 01:48:40.272123 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:40.272090 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert podName:f2d2e933-102a-44e0-9813-093c6ff90178 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:41.272073024 +0000 UTC m=+60.949497113 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-b76q4" (UID: "f2d2e933-102a-44e0-9813-093c6ff90178") : secret "networking-console-plugin-cert" not found Jul 18 01:48:40.326058 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:40.326030 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 01:48:40.326423 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:40.326389 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/0.log" Jul 18 01:48:40.326465 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:40.326421 2570 generic.go:356] "Generic (PLEG): container finished" podID="d5439579-cb04-47d9-99a6-60c482c4afbd" containerID="7518642d4a360d56eb500aca405f377cf7e0ac26d54cde803c2330eedf7cd8e0" exitCode=255 Jul 18 01:48:40.326551 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:40.326523 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-8m75k" event={"ID":"d5439579-cb04-47d9-99a6-60c482c4afbd","Type":"ContainerDied","Data":"7518642d4a360d56eb500aca405f377cf7e0ac26d54cde803c2330eedf7cd8e0"} Jul 18 01:48:40.326626 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:40.326612 2570 scope.go:117] "RemoveContainer" containerID="5cb762874f6041806f2b89d6d2c7df4873a6859d6a40c48a602c55d5680a6c4d" Jul 18 01:48:40.326756 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:40.326741 2570 scope.go:117] "RemoveContainer" containerID="7518642d4a360d56eb500aca405f377cf7e0ac26d54cde803c2330eedf7cd8e0" Jul 18 01:48:40.326930 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:40.326908 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-8m75k_openshift-console-operator(d5439579-cb04-47d9-99a6-60c482c4afbd)\"" pod="openshift-console-operator/console-operator-565b7b569d-8m75k" podUID="d5439579-cb04-47d9-99a6-60c482c4afbd" Jul 18 01:48:41.279918 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.279865 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-b76q4\" (UID: \"f2d2e933-102a-44e0-9813-093c6ff90178\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4" Jul 18 01:48:41.280109 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:41.280031 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 18 01:48:41.280109 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:41.280098 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert podName:f2d2e933-102a-44e0-9813-093c6ff90178 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:43.280082454 +0000 UTC m=+62.957506562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-b76q4" (UID: "f2d2e933-102a-44e0-9813-093c6ff90178") : secret "networking-console-plugin-cert" not found Jul 18 01:48:41.330342 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.330307 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 01:48:41.330718 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.330659 2570 scope.go:117] "RemoveContainer" containerID="7518642d4a360d56eb500aca405f377cf7e0ac26d54cde803c2330eedf7cd8e0" Jul 18 01:48:41.330820 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:41.330800 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-8m75k_openshift-console-operator(d5439579-cb04-47d9-99a6-60c482c4afbd)\"" pod="openshift-console-operator/console-operator-565b7b569d-8m75k" podUID="d5439579-cb04-47d9-99a6-60c482c4afbd" Jul 18 01:48:41.687898 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.687868 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kh7bp"] Jul 18 01:48:41.692396 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.692378 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:41.694407 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.694390 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 18 01:48:41.694500 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.694406 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 18 01:48:41.694684 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.694665 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zn9g6\"" Jul 18 01:48:41.694784 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.694721 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 18 01:48:41.694831 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.694801 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 18 01:48:41.699606 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.699585 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kh7bp"] Jul 18 01:48:41.783392 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.783357 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:41.783579 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.783422 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-data-volume\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:41.783579 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.783537 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwl9j\" (UniqueName: \"kubernetes.io/projected/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-api-access-gwl9j\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:41.783723 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.783594 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-crio-socket\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:41.783723 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.783705 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:41.884076 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.884038 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:41.884268 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.884084 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:41.884268 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.884120 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-data-volume\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:41.884268 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.884164 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwl9j\" (UniqueName: \"kubernetes.io/projected/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-api-access-gwl9j\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:41.884268 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:41.884168 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 01:48:41.884268 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.884201 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-crio-socket\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:41.884268 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:41.884247 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm podName:9d0a5939-81a2-4ed5-a54a-84ed81158dd5 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:42.384232051 +0000 UTC m=+62.061656140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kh7bp" (UID: "9d0a5939-81a2-4ed5-a54a-84ed81158dd5") : configmap "kube-rbac-proxy" not found Jul 18 01:48:41.884596 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.884299 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-crio-socket\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:41.884596 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:41.884339 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:48:41.884596 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:41.884404 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls podName:9d0a5939-81a2-4ed5-a54a-84ed81158dd5 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:42.384387219 +0000 UTC m=+62.061811324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kh7bp" (UID: "9d0a5939-81a2-4ed5-a54a-84ed81158dd5") : secret "insights-runtime-extractor-tls" not found Jul 18 01:48:41.884596 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.884554 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-data-volume\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:41.891813 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:41.891790 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwl9j\" (UniqueName: \"kubernetes.io/projected/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-api-access-gwl9j\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:42.245673 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.245637 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jshl6" Jul 18 01:48:42.387863 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.387827 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:42.388270 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.387871 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:42.388270 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:42.387943 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 01:48:42.388270 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:42.387968 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:48:42.388270 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:42.388010 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm podName:9d0a5939-81a2-4ed5-a54a-84ed81158dd5 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:43.387992236 +0000 UTC m=+63.065416341 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kh7bp" (UID: "9d0a5939-81a2-4ed5-a54a-84ed81158dd5") : configmap "kube-rbac-proxy" not found Jul 18 01:48:42.388270 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:42.388024 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls podName:9d0a5939-81a2-4ed5-a54a-84ed81158dd5 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:43.388018731 +0000 UTC m=+63.065442820 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kh7bp" (UID: "9d0a5939-81a2-4ed5-a54a-84ed81158dd5") : secret "insights-runtime-extractor-tls" not found Jul 18 01:48:42.427456 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.427424 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-8sstg"] Jul 18 01:48:42.430483 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.430467 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-8sstg" Jul 18 01:48:42.432754 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.432724 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 18 01:48:42.432881 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.432797 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 18 01:48:42.432881 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.432852 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-s2tgj\"" Jul 18 01:48:42.432881 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.432854 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 18 01:48:42.433011 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.432797 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 18 01:48:42.438910 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.438887 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-8sstg"] Jul 18 01:48:42.488585 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.488554 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/45dcc625-759b-400d-b1a4-792353f1dc6d-signing-key\") pod \"service-ca-58598695f7-8sstg\" (UID: \"45dcc625-759b-400d-b1a4-792353f1dc6d\") " pod="openshift-service-ca/service-ca-58598695f7-8sstg" Jul 18 01:48:42.488585 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.488585 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/45dcc625-759b-400d-b1a4-792353f1dc6d-signing-cabundle\") pod \"service-ca-58598695f7-8sstg\" (UID: \"45dcc625-759b-400d-b1a4-792353f1dc6d\") " pod="openshift-service-ca/service-ca-58598695f7-8sstg" Jul 18 01:48:42.488790 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.488726 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xfvg\" (UniqueName: \"kubernetes.io/projected/45dcc625-759b-400d-b1a4-792353f1dc6d-kube-api-access-8xfvg\") pod \"service-ca-58598695f7-8sstg\" (UID: \"45dcc625-759b-400d-b1a4-792353f1dc6d\") " pod="openshift-service-ca/service-ca-58598695f7-8sstg" Jul 18 01:48:42.589868 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.589835 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xfvg\" (UniqueName: \"kubernetes.io/projected/45dcc625-759b-400d-b1a4-792353f1dc6d-kube-api-access-8xfvg\") pod \"service-ca-58598695f7-8sstg\" (UID: \"45dcc625-759b-400d-b1a4-792353f1dc6d\") " pod="openshift-service-ca/service-ca-58598695f7-8sstg" Jul 18 01:48:42.590034 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.589915 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/45dcc625-759b-400d-b1a4-792353f1dc6d-signing-key\") pod \"service-ca-58598695f7-8sstg\" (UID: \"45dcc625-759b-400d-b1a4-792353f1dc6d\") " pod="openshift-service-ca/service-ca-58598695f7-8sstg" Jul 18 01:48:42.590034 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.589943 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/45dcc625-759b-400d-b1a4-792353f1dc6d-signing-cabundle\") pod \"service-ca-58598695f7-8sstg\" (UID: \"45dcc625-759b-400d-b1a4-792353f1dc6d\") " pod="openshift-service-ca/service-ca-58598695f7-8sstg" Jul 18 01:48:42.590581 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.590562 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/45dcc625-759b-400d-b1a4-792353f1dc6d-signing-cabundle\") pod \"service-ca-58598695f7-8sstg\" (UID: \"45dcc625-759b-400d-b1a4-792353f1dc6d\") " pod="openshift-service-ca/service-ca-58598695f7-8sstg" Jul 18 01:48:42.592306 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.592272 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/45dcc625-759b-400d-b1a4-792353f1dc6d-signing-key\") pod \"service-ca-58598695f7-8sstg\" (UID: \"45dcc625-759b-400d-b1a4-792353f1dc6d\") " pod="openshift-service-ca/service-ca-58598695f7-8sstg" Jul 18 01:48:42.599441 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.599414 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xfvg\" (UniqueName: \"kubernetes.io/projected/45dcc625-759b-400d-b1a4-792353f1dc6d-kube-api-access-8xfvg\") pod \"service-ca-58598695f7-8sstg\" (UID: \"45dcc625-759b-400d-b1a4-792353f1dc6d\") " pod="openshift-service-ca/service-ca-58598695f7-8sstg" Jul 18 01:48:42.739894 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.739837 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-8sstg" Jul 18 01:48:42.860251 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:42.860173 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-8sstg"] Jul 18 01:48:42.863714 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:48:42.863692 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45dcc625_759b_400d_b1a4_792353f1dc6d.slice/crio-430703e01e568d7db5ae446fe1e2239360a62c9a4e4fcbc974d99dbe997d5e3d WatchSource:0}: Error finding container 430703e01e568d7db5ae446fe1e2239360a62c9a4e4fcbc974d99dbe997d5e3d: Status 404 returned error can't find the container with id 430703e01e568d7db5ae446fe1e2239360a62c9a4e4fcbc974d99dbe997d5e3d Jul 18 01:48:43.165667 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:43.165589 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8xjzh_270e5f34-05a1-4e34-828a-2ade34235111/dns-node-resolver/0.log" Jul 18 01:48:43.296116 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:43.296075 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-b76q4\" (UID: \"f2d2e933-102a-44e0-9813-093c6ff90178\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4" Jul 18 01:48:43.296288 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:43.296220 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 18 01:48:43.296331 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:43.296291 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert podName:f2d2e933-102a-44e0-9813-093c6ff90178 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:47.296266497 +0000 UTC m=+66.973690594 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-b76q4" (UID: "f2d2e933-102a-44e0-9813-093c6ff90178") : secret "networking-console-plugin-cert" not found Jul 18 01:48:43.337117 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:43.337080 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-8sstg" event={"ID":"45dcc625-759b-400d-b1a4-792353f1dc6d","Type":"ContainerStarted","Data":"9440df977096dbe0069e006ad1f71ec86af049215fa0db898e276e499745548b"} Jul 18 01:48:43.337271 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:43.337139 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-8sstg" event={"ID":"45dcc625-759b-400d-b1a4-792353f1dc6d","Type":"ContainerStarted","Data":"430703e01e568d7db5ae446fe1e2239360a62c9a4e4fcbc974d99dbe997d5e3d"} Jul 18 01:48:43.367245 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:43.367192 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-8sstg" podStartSLOduration=1.367172698 podStartE2EDuration="1.367172698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:43.365046367 +0000 UTC m=+63.042470477" watchObservedRunningTime="2026-07-18 01:48:43.367172698 +0000 UTC m=+63.044596811" Jul 18 01:48:43.397257 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:43.397232 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:43.397659 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:43.397284 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:43.397659 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:43.397362 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 01:48:43.397659 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:43.397422 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm podName:9d0a5939-81a2-4ed5-a54a-84ed81158dd5 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:45.397405371 +0000 UTC m=+65.074829473 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kh7bp" (UID: "9d0a5939-81a2-4ed5-a54a-84ed81158dd5") : configmap "kube-rbac-proxy" not found Jul 18 01:48:43.397659 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:43.397493 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:48:43.397659 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:43.397549 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls podName:9d0a5939-81a2-4ed5-a54a-84ed81158dd5 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:45.397533644 +0000 UTC m=+65.074957756 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kh7bp" (UID: "9d0a5939-81a2-4ed5-a54a-84ed81158dd5") : secret "insights-runtime-extractor-tls" not found Jul 18 01:48:43.801327 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:43.801294 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-rh4q8\" (UID: \"e8e164cc-e6b4-4ebd-b122-e383719192d0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" Jul 18 01:48:43.801485 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:43.801421 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:43.801527 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:43.801504 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls podName:e8e164cc-e6b4-4ebd-b122-e383719192d0 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:51.801483545 +0000 UTC m=+71.478907641 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-rh4q8" (UID: "e8e164cc-e6b4-4ebd-b122-e383719192d0") : secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:44.165231 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:44.165152 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-566ms_abf94060-f8ec-42d6-8e79-2c6d3490ebf8/node-ca/0.log" Jul 18 01:48:44.981334 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:44.981301 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b78nn_f9231edc-64b9-43d0-b939-e9043f91e610/serve-healthcheck-canary/0.log" Jul 18 01:48:45.416239 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:45.416193 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:45.416455 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:45.416255 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:45.416455 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:45.416332 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 01:48:45.416455 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:45.416411 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm podName:9d0a5939-81a2-4ed5-a54a-84ed81158dd5 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:49.416390832 +0000 UTC m=+69.093814921 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kh7bp" (UID: "9d0a5939-81a2-4ed5-a54a-84ed81158dd5") : configmap "kube-rbac-proxy" not found Jul 18 01:48:45.416455 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:45.416410 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:48:45.416455 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:45.416446 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls podName:9d0a5939-81a2-4ed5-a54a-84ed81158dd5 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:49.416437959 +0000 UTC m=+69.093862051 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kh7bp" (UID: "9d0a5939-81a2-4ed5-a54a-84ed81158dd5") : secret "insights-runtime-extractor-tls" not found Jul 18 01:48:45.718636 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:45.718541 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret\") pod \"global-pull-secret-syncer-bdkrt\" (UID: \"2693ad6a-f241-4165-8d2e-9aee83cc8ae8\") " pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:48:45.718636 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:45.718610 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs\") pod \"network-metrics-daemon-5nqdq\" (UID: \"bf7391f7-3b4a-4eb5-ab3a-b540543d4907\") " pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:48:45.720986 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:45.720965 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 01:48:45.721084 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:45.720992 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 01:48:45.729685 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:45.729666 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 18 01:48:45.729771 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:45.729732 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs podName:bf7391f7-3b4a-4eb5-ab3a-b540543d4907 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:49.72971189 +0000 UTC m=+129.407135992 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs") pod "network-metrics-daemon-5nqdq" (UID: "bf7391f7-3b4a-4eb5-ab3a-b540543d4907") : secret "metrics-daemon-secret" not found Jul 18 01:48:45.731743 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:45.731727 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2693ad6a-f241-4165-8d2e-9aee83cc8ae8-original-pull-secret\") pod \"global-pull-secret-syncer-bdkrt\" (UID: \"2693ad6a-f241-4165-8d2e-9aee83cc8ae8\") " pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:48:45.819101 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:45.819068 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwchk\" (UniqueName: \"kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk\") pod \"network-check-target-2shnl\" (UID: \"af86de24-1529-4faa-9cdb-48c4cf6d6ede\") " pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:48:45.821778 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:45.821759 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 18 01:48:45.832106 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:45.832090 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 18 01:48:45.842365 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:45.842335 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwchk\" (UniqueName: \"kubernetes.io/projected/af86de24-1529-4faa-9cdb-48c4cf6d6ede-kube-api-access-qwchk\") pod \"network-check-target-2shnl\" (UID: \"af86de24-1529-4faa-9cdb-48c4cf6d6ede\") " pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:48:45.898094 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:45.898066 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bdkrt" Jul 18 01:48:45.905350 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:45.905326 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vwvlx\"" Jul 18 01:48:45.914203 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:45.914179 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:48:46.033503 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:46.033400 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bdkrt"] Jul 18 01:48:46.036319 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:48:46.036285 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2693ad6a_f241_4165_8d2e_9aee83cc8ae8.slice/crio-507d518e63d41441d0d4388f4dac5c7cefc1461be61da2a0d2a8919575593c52 WatchSource:0}: Error finding container 507d518e63d41441d0d4388f4dac5c7cefc1461be61da2a0d2a8919575593c52: Status 404 returned error can't find the container with id 507d518e63d41441d0d4388f4dac5c7cefc1461be61da2a0d2a8919575593c52 Jul 18 01:48:46.049389 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:46.049364 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2shnl"] Jul 18 01:48:46.053040 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:48:46.053016 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf86de24_1529_4faa_9cdb_48c4cf6d6ede.slice/crio-760d80e6b6faca326701022f67ec43109eefbb2c7034a7f5de5138bf195a2665 WatchSource:0}: Error finding container 760d80e6b6faca326701022f67ec43109eefbb2c7034a7f5de5138bf195a2665: Status 404 returned error can't find the container with id 760d80e6b6faca326701022f67ec43109eefbb2c7034a7f5de5138bf195a2665 Jul 18 01:48:46.181342 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:46.181303 2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:48:46.181503 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:46.181368 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:48:46.181759 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:46.181745 2570 scope.go:117] "RemoveContainer" containerID="7518642d4a360d56eb500aca405f377cf7e0ac26d54cde803c2330eedf7cd8e0" Jul 18 01:48:46.181911 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:46.181892 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-8m75k_openshift-console-operator(d5439579-cb04-47d9-99a6-60c482c4afbd)\"" pod="openshift-console-operator/console-operator-565b7b569d-8m75k" podUID="d5439579-cb04-47d9-99a6-60c482c4afbd" Jul 18 01:48:46.346493 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:46.346457 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bdkrt" event={"ID":"2693ad6a-f241-4165-8d2e-9aee83cc8ae8","Type":"ContainerStarted","Data":"507d518e63d41441d0d4388f4dac5c7cefc1461be61da2a0d2a8919575593c52"} Jul 18 01:48:46.347538 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:46.347518 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2shnl" event={"ID":"af86de24-1529-4faa-9cdb-48c4cf6d6ede","Type":"ContainerStarted","Data":"760d80e6b6faca326701022f67ec43109eefbb2c7034a7f5de5138bf195a2665"} Jul 18 01:48:47.331839 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:47.331619 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-b76q4\" (UID: \"f2d2e933-102a-44e0-9813-093c6ff90178\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4" Jul 18 01:48:47.331839 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:47.331795 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 18 01:48:47.332516 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:47.331878 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert podName:f2d2e933-102a-44e0-9813-093c6ff90178 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:55.331855932 +0000 UTC m=+75.009280036 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-b76q4" (UID: "f2d2e933-102a-44e0-9813-093c6ff90178") : secret "networking-console-plugin-cert" not found Jul 18 01:48:47.635382 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:47.635296 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:48:47.635561 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:47.635462 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:48:47.635561 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:47.635478 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5847499567-xn4cp: secret "image-registry-tls" not found Jul 18 01:48:47.635561 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:47.635532 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls podName:632a8dc2-b00b-419a-a1f7-416a4cfa380c nodeName:}" failed. No retries permitted until 2026-07-18 01:49:19.635515801 +0000 UTC m=+99.312939904 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls") pod "image-registry-5847499567-xn4cp" (UID: "632a8dc2-b00b-419a-a1f7-416a4cfa380c") : secret "image-registry-tls" not found Jul 18 01:48:47.736656 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:47.736603 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:48:47.736848 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:47.736774 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:48:47.736915 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:47.736861 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls podName:163a2e0f-86f1-4c36-ba43-6d097cbec349 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:19.736839943 +0000 UTC m=+99.414264051 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls") pod "dns-default-bzrjc" (UID: "163a2e0f-86f1-4c36-ba43-6d097cbec349") : secret "dns-default-metrics-tls" not found Jul 18 01:48:49.453442 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:49.453384 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:49.453926 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:49.453454 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:49.453926 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:49.453533 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 01:48:49.453926 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:49.453613 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm podName:9d0a5939-81a2-4ed5-a54a-84ed81158dd5 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:57.453591764 +0000 UTC m=+77.131015858 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kh7bp" (UID: "9d0a5939-81a2-4ed5-a54a-84ed81158dd5") : configmap "kube-rbac-proxy" not found Jul 18 01:48:49.453926 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:49.453617 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:48:49.453926 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:48:49.453684 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls podName:9d0a5939-81a2-4ed5-a54a-84ed81158dd5 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:57.453668433 +0000 UTC m=+77.131092528 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kh7bp" (UID: "9d0a5939-81a2-4ed5-a54a-84ed81158dd5") : secret "insights-runtime-extractor-tls" not found Jul 18 01:48:51.367717 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:51.367683 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2shnl" event={"ID":"af86de24-1529-4faa-9cdb-48c4cf6d6ede","Type":"ContainerStarted","Data":"7fa66db6bd5541fd7c392cc1652cbeb7c198e1b899fe7ed863c486331e73cf3c"} Jul 18 01:48:51.368116 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:51.367832 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:48:51.381822 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:51.381773 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2shnl" podStartSLOduration=65.21748133 podStartE2EDuration="1m10.381750517s" totalImagesPullingTime="5.164269187s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:51.381733436 +0000 UTC m=+71.059157550" watchObservedRunningTime="2026-07-18 01:48:51.381750517 +0000 UTC m=+71.059174609" Jul 18 01:48:51.875868 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:51.875828 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-rh4q8\" (UID: \"e8e164cc-e6b4-4ebd-b122-e383719192d0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" Jul 18 01:48:51.878203 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:51.878182 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8e164cc-e6b4-4ebd-b122-e383719192d0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-rh4q8\" (UID: \"e8e164cc-e6b4-4ebd-b122-e383719192d0\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" Jul 18 01:48:51.971022 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:51.970992 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-s7k8d\"" Jul 18 01:48:51.979037 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:51.979000 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" Jul 18 01:48:52.136234 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:52.136159 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8"] Jul 18 01:48:52.139763 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:48:52.139740 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8e164cc_e6b4_4ebd_b122_e383719192d0.slice/crio-e46202e3fac8e69caba4b07592f64e8319e2e60c9bd342b95837d8036599febd WatchSource:0}: Error finding container e46202e3fac8e69caba4b07592f64e8319e2e60c9bd342b95837d8036599febd: Status 404 returned error can't find the container with id e46202e3fac8e69caba4b07592f64e8319e2e60c9bd342b95837d8036599febd Jul 18 01:48:52.372255 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:52.372208 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bdkrt" event={"ID":"2693ad6a-f241-4165-8d2e-9aee83cc8ae8","Type":"ContainerStarted","Data":"6a3e3c01c3f380adababb6bd7d2b4b677a1ceb8d9e0ed15a201aecf246cc9112"} Jul 18 01:48:52.373461 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:52.373436 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" event={"ID":"e8e164cc-e6b4-4ebd-b122-e383719192d0","Type":"ContainerStarted","Data":"e46202e3fac8e69caba4b07592f64e8319e2e60c9bd342b95837d8036599febd"} Jul 18 01:48:52.388305 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:52.388190 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bdkrt" podStartSLOduration=66.196113903 podStartE2EDuration="1m11.388173374s" totalImagesPullingTime="5.192059471s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:52.387740537 +0000 UTC m=+72.065164672" watchObservedRunningTime="2026-07-18 01:48:52.388173374 +0000 UTC m=+72.065597485" Jul 18 01:48:54.381951 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:54.381919 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" event={"ID":"e8e164cc-e6b4-4ebd-b122-e383719192d0","Type":"ContainerStarted","Data":"44ce431cfe24469b315256c3a9f5c1469b8e0b84610a4c5b763beb2292d45d27"} Jul 18 01:48:54.403257 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:54.403204 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-rh4q8" podStartSLOduration=16.519333239 podStartE2EDuration="18.403186933s" totalImagesPullingTime="1.883853694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:54.402030112 +0000 UTC m=+74.079454223" watchObservedRunningTime="2026-07-18 01:48:54.403186933 +0000 UTC m=+74.080611043" Jul 18 01:48:55.407317 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:55.407281 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-b76q4\" (UID: \"f2d2e933-102a-44e0-9813-093c6ff90178\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4" Jul 18 01:48:55.409671 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:55.409651 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/f2d2e933-102a-44e0-9813-093c6ff90178-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-b76q4\" (UID: \"f2d2e933-102a-44e0-9813-093c6ff90178\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4" Jul 18 01:48:55.537258 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:55.537224 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-g72gf\"" Jul 18 01:48:55.546096 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:55.546069 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4" Jul 18 01:48:55.670154 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:55.670127 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4"] Jul 18 01:48:55.672604 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:48:55.672578 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2d2e933_102a_44e0_9813_093c6ff90178.slice/crio-1e831a8e513082f089dd44964e7d39be1fd171a548fabed7357c48d5236de684 WatchSource:0}: Error finding container 1e831a8e513082f089dd44964e7d39be1fd171a548fabed7357c48d5236de684: Status 404 returned error can't find the container with id 1e831a8e513082f089dd44964e7d39be1fd171a548fabed7357c48d5236de684 Jul 18 01:48:56.390070 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:56.390026 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4" event={"ID":"f2d2e933-102a-44e0-9813-093c6ff90178","Type":"ContainerStarted","Data":"1e831a8e513082f089dd44964e7d39be1fd171a548fabed7357c48d5236de684"} Jul 18 01:48:57.394924 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:57.394889 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4" event={"ID":"f2d2e933-102a-44e0-9813-093c6ff90178","Type":"ContainerStarted","Data":"017fcf709fa1354d6a7252d06a5ec84edb2f1c350eb87f69bf55162708b4b9cf"} Jul 18 01:48:57.413513 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:57.413466 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-b76q4" podStartSLOduration=17.177274201 podStartE2EDuration="18.41345201s" totalImagesPullingTime="1.236177809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:57.41312448 +0000 UTC m=+77.090548600" watchObservedRunningTime="2026-07-18 01:48:57.41345201 +0000 UTC m=+77.090876165" Jul 18 01:48:57.526943 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:57.526904 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:57.527207 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:57.527181 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:57.527685 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:57.527669 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:57.529215 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:57.529197 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d0a5939-81a2-4ed5-a54a-84ed81158dd5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kh7bp\" (UID: \"9d0a5939-81a2-4ed5-a54a-84ed81158dd5\") " pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:57.602415 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:57.602380 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kh7bp" Jul 18 01:48:57.721592 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:57.721560 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kh7bp"] Jul 18 01:48:57.724496 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:48:57.724461 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d0a5939_81a2_4ed5_a54a_84ed81158dd5.slice/crio-253175f78062044742312b8c9e25be9f6c81d5ceddf8ee80aa654b61ee508028 WatchSource:0}: Error finding container 253175f78062044742312b8c9e25be9f6c81d5ceddf8ee80aa654b61ee508028: Status 404 returned error can't find the container with id 253175f78062044742312b8c9e25be9f6c81d5ceddf8ee80aa654b61ee508028 Jul 18 01:48:58.400981 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:58.400927 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kh7bp" event={"ID":"9d0a5939-81a2-4ed5-a54a-84ed81158dd5","Type":"ContainerStarted","Data":"253175f78062044742312b8c9e25be9f6c81d5ceddf8ee80aa654b61ee508028"} Jul 18 01:48:59.889212 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:48:59.889184 2570 scope.go:117] "RemoveContainer" containerID="7518642d4a360d56eb500aca405f377cf7e0ac26d54cde803c2330eedf7cd8e0" Jul 18 01:49:03.424058 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:03.423670 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 01:49:03.424058 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:03.423805 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-8m75k" event={"ID":"d5439579-cb04-47d9-99a6-60c482c4afbd","Type":"ContainerStarted","Data":"da02b626be9cdaf42c9570e1050a731366bf296faf0fcd2f351c61180dd5cdf9"} Jul 18 01:49:03.424683 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:03.424628 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:49:03.426887 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:03.426858 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kh7bp" event={"ID":"9d0a5939-81a2-4ed5-a54a-84ed81158dd5","Type":"ContainerStarted","Data":"eb34ba36b4066f33a2fbb2ea451e476bf64d28cd91ef90bda5f8012f576676d2"} Jul 18 01:49:03.445631 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:03.445530 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-8m75k" podStartSLOduration=25.950311896 podStartE2EDuration="28.445510494s" totalImagesPullingTime="2.495198598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:03.444129928 +0000 UTC m=+83.121554039" watchObservedRunningTime="2026-07-18 01:49:03.445510494 +0000 UTC m=+83.122934648" Jul 18 01:49:03.649346 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:03.649281 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-8m75k" Jul 18 01:49:05.651774 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.651740 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wjc9q"] Jul 18 01:49:05.663934 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.663894 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wjc9q" Jul 18 01:49:05.667292 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.667267 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-kxctv"] Jul 18 01:49:05.669813 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.669786 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-hs9nv\"" Jul 18 01:49:05.669970 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.669951 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 18 01:49:05.672210 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.672191 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-kxctv" Jul 18 01:49:05.674985 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.674965 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 18 01:49:05.675087 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.675005 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-4z4n8\"" Jul 18 01:49:05.675786 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.675766 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 18 01:49:05.677093 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.677070 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wjc9q"] Jul 18 01:49:05.688878 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.688854 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-kxctv"] Jul 18 01:49:05.797504 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.797466 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3150d481-c72f-4cca-b9ba-049e43ddd627-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-wjc9q\" (UID: \"3150d481-c72f-4cca-b9ba-049e43ddd627\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wjc9q" Jul 18 01:49:05.797692 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.797547 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vhqj\" (UniqueName: \"kubernetes.io/projected/41d8c727-0f8f-4fdf-ae90-c10fd2caaaff-kube-api-access-5vhqj\") pod \"downloads-5bc67dcd46-kxctv\" (UID: \"41d8c727-0f8f-4fdf-ae90-c10fd2caaaff\") " pod="openshift-console/downloads-5bc67dcd46-kxctv" Jul 18 01:49:05.898725 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.898699 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vhqj\" (UniqueName: \"kubernetes.io/projected/41d8c727-0f8f-4fdf-ae90-c10fd2caaaff-kube-api-access-5vhqj\") pod \"downloads-5bc67dcd46-kxctv\" (UID: \"41d8c727-0f8f-4fdf-ae90-c10fd2caaaff\") " pod="openshift-console/downloads-5bc67dcd46-kxctv" Jul 18 01:49:05.898890 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.898786 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3150d481-c72f-4cca-b9ba-049e43ddd627-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-wjc9q\" (UID: \"3150d481-c72f-4cca-b9ba-049e43ddd627\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wjc9q" Jul 18 01:49:05.901446 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.901417 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3150d481-c72f-4cca-b9ba-049e43ddd627-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-wjc9q\" (UID: \"3150d481-c72f-4cca-b9ba-049e43ddd627\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wjc9q" Jul 18 01:49:05.914731 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.914621 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vhqj\" (UniqueName: \"kubernetes.io/projected/41d8c727-0f8f-4fdf-ae90-c10fd2caaaff-kube-api-access-5vhqj\") pod \"downloads-5bc67dcd46-kxctv\" (UID: \"41d8c727-0f8f-4fdf-ae90-c10fd2caaaff\") " pod="openshift-console/downloads-5bc67dcd46-kxctv" Jul 18 01:49:05.975293 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.975260 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wjc9q" Jul 18 01:49:05.983294 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:05.983278 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-kxctv" Jul 18 01:49:06.162473 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:49:06.162447 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41d8c727_0f8f_4fdf_ae90_c10fd2caaaff.slice/crio-cdfc4408878d308f91604c54dd59171824b3459471a5daf068f3363ab6ee1c96 WatchSource:0}: Error finding container cdfc4408878d308f91604c54dd59171824b3459471a5daf068f3363ab6ee1c96: Status 404 returned error can't find the container with id cdfc4408878d308f91604c54dd59171824b3459471a5daf068f3363ab6ee1c96 Jul 18 01:49:06.170993 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:06.170973 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-kxctv"] Jul 18 01:49:06.177587 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:06.177563 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wjc9q"] Jul 18 01:49:06.181084 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:49:06.181061 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3150d481_c72f_4cca_b9ba_049e43ddd627.slice/crio-6c56ca5c1db28f46b9ac79ecf7cb798a1ef089040d0d3bbcc0f39fc7dd72036e WatchSource:0}: Error finding container 6c56ca5c1db28f46b9ac79ecf7cb798a1ef089040d0d3bbcc0f39fc7dd72036e: Status 404 returned error can't find the container with id 6c56ca5c1db28f46b9ac79ecf7cb798a1ef089040d0d3bbcc0f39fc7dd72036e Jul 18 01:49:06.442178 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:06.442100 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-kxctv" event={"ID":"41d8c727-0f8f-4fdf-ae90-c10fd2caaaff","Type":"ContainerStarted","Data":"cdfc4408878d308f91604c54dd59171824b3459471a5daf068f3363ab6ee1c96"} Jul 18 01:49:06.443470 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:06.443423 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wjc9q" event={"ID":"3150d481-c72f-4cca-b9ba-049e43ddd627","Type":"ContainerStarted","Data":"6c56ca5c1db28f46b9ac79ecf7cb798a1ef089040d0d3bbcc0f39fc7dd72036e"} Jul 18 01:49:06.445477 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:06.445454 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kh7bp" event={"ID":"9d0a5939-81a2-4ed5-a54a-84ed81158dd5","Type":"ContainerStarted","Data":"3455dfe67dd3a5e4170381f63fd2f442b9ecd5fd75e9b38fef3940c00da7170a"} Jul 18 01:49:08.455430 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:08.455389 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wjc9q" event={"ID":"3150d481-c72f-4cca-b9ba-049e43ddd627","Type":"ContainerStarted","Data":"7ec0b9575b40a2420418727ec153b08ad37a7ad767c5e90c362f8effcb3a0935"} Jul 18 01:49:08.455870 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:08.455823 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wjc9q" Jul 18 01:49:08.462772 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:08.462748 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wjc9q" Jul 18 01:49:08.473597 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:08.473433 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-wjc9q" podStartSLOduration=2.165851891 podStartE2EDuration="3.47341547s" totalImagesPullingTime="1.307563579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:08.473232949 +0000 UTC m=+88.150657063" watchObservedRunningTime="2026-07-18 01:49:08.47341547 +0000 UTC m=+88.150839585" Jul 18 01:49:10.465561 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:10.465519 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kh7bp" event={"ID":"9d0a5939-81a2-4ed5-a54a-84ed81158dd5","Type":"ContainerStarted","Data":"c9ede6b3d2255836d110e8e7e94b6288aa40a219addedfcfff8d4677df561d1e"} Jul 18 01:49:10.484734 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:10.484686 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kh7bp" podStartSLOduration=17.372581519 podStartE2EDuration="29.484672155s" totalImagesPullingTime="12.112090636s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:10.483229236 +0000 UTC m=+90.160653347" watchObservedRunningTime="2026-07-18 01:49:10.484672155 +0000 UTC m=+90.162096265" Jul 18 01:49:17.966688 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:17.965684 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-c68mx"] Jul 18 01:49:17.970073 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:17.970038 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:17.973099 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:17.972910 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 18 01:49:17.973219 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:17.973174 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 18 01:49:17.973595 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:17.973413 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 18 01:49:17.973595 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:17.973458 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jzc4s\"" Jul 18 01:49:17.973979 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:17.973962 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 18 01:49:17.991524 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:17.991501 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/31d64b4a-8386-4a23-be41-0df94f6cabf4-root\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:17.991627 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:17.991542 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:17.991627 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:17.991610 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-accelerators-collector-config\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:17.991767 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:17.991691 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-tls\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:17.991767 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:17.991752 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/31d64b4a-8386-4a23-be41-0df94f6cabf4-sys\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:17.991866 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:17.991775 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-textfile\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:17.991866 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:17.991816 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-775qg\" (UniqueName: \"kubernetes.io/projected/31d64b4a-8386-4a23-be41-0df94f6cabf4-kube-api-access-775qg\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:17.991866 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:17.991848 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31d64b4a-8386-4a23-be41-0df94f6cabf4-metrics-client-ca\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:17.992005 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:17.991876 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-wtmp\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.092655 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.092607 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/31d64b4a-8386-4a23-be41-0df94f6cabf4-root\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.092833 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.092679 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.092833 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.092721 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-accelerators-collector-config\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.092833 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.092745 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-tls\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.092833 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.092786 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/31d64b4a-8386-4a23-be41-0df94f6cabf4-sys\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.092833 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.092802 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-textfile\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.092833 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.092826 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-775qg\" (UniqueName: \"kubernetes.io/projected/31d64b4a-8386-4a23-be41-0df94f6cabf4-kube-api-access-775qg\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.093149 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.092844 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31d64b4a-8386-4a23-be41-0df94f6cabf4-metrics-client-ca\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.093149 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.092864 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-wtmp\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.093149 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.092985 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-wtmp\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.093149 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:49:18.093070 2570 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 18 01:49:18.093149 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:49:18.093110 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-tls podName:31d64b4a-8386-4a23-be41-0df94f6cabf4 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:18.593097829 +0000 UTC m=+98.270521918 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-tls") pod "node-exporter-c68mx" (UID: "31d64b4a-8386-4a23-be41-0df94f6cabf4") : secret "node-exporter-tls" not found Jul 18 01:49:18.093405 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.092742 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/31d64b4a-8386-4a23-be41-0df94f6cabf4-root\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.093405 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.093317 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/31d64b4a-8386-4a23-be41-0df94f6cabf4-sys\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.093405 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.093394 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-accelerators-collector-config\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.093616 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.093500 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-textfile\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.094235 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.094180 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/31d64b4a-8386-4a23-be41-0df94f6cabf4-metrics-client-ca\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.096460 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.096369 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.102108 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.102084 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-775qg\" (UniqueName: \"kubernetes.io/projected/31d64b4a-8386-4a23-be41-0df94f6cabf4-kube-api-access-775qg\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.597332 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.597291 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-tls\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.599801 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.599776 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/31d64b4a-8386-4a23-be41-0df94f6cabf4-node-exporter-tls\") pod \"node-exporter-c68mx\" (UID: \"31d64b4a-8386-4a23-be41-0df94f6cabf4\") " pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:18.882681 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:18.882530 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-c68mx" Jul 18 01:49:19.709730 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:19.709692 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:49:19.712441 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:19.712411 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls\") pod \"image-registry-5847499567-xn4cp\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:49:19.810523 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:19.810487 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:49:19.813180 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:19.813145 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/163a2e0f-86f1-4c36-ba43-6d097cbec349-metrics-tls\") pod \"dns-default-bzrjc\" (UID: \"163a2e0f-86f1-4c36-ba43-6d097cbec349\") " pod="openshift-dns/dns-default-bzrjc" Jul 18 01:49:19.904670 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:19.904621 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-66smx\"" Jul 18 01:49:19.912595 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:19.912566 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:49:20.019976 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:20.019898 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ml5qw\"" Jul 18 01:49:20.027919 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:20.027895 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bzrjc" Jul 18 01:49:22.325356 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.325322 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-664cc87cf7-glvjn"] Jul 18 01:49:22.358466 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.358438 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-664cc87cf7-glvjn"] Jul 18 01:49:22.358630 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.358552 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.362382 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.362248 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 18 01:49:22.362382 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.362249 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 18 01:49:22.362382 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.362294 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 18 01:49:22.362382 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.362249 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 18 01:49:22.362705 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.362453 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8e57uv0bgb35p\"" Jul 18 01:49:22.362705 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.362473 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-wpv58\"" Jul 18 01:49:22.375861 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.375838 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2shnl" Jul 18 01:49:22.431484 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.431453 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/971cbd12-3157-444a-b7a4-fbb12ff4db1c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.431666 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.431501 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/971cbd12-3157-444a-b7a4-fbb12ff4db1c-secret-metrics-server-client-certs\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.431666 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.431529 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/971cbd12-3157-444a-b7a4-fbb12ff4db1c-metrics-server-audit-profiles\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.431666 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.431588 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhffk\" (UniqueName: \"kubernetes.io/projected/971cbd12-3157-444a-b7a4-fbb12ff4db1c-kube-api-access-rhffk\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.431666 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.431621 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/971cbd12-3157-444a-b7a4-fbb12ff4db1c-secret-metrics-server-tls\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.431887 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.431677 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/971cbd12-3157-444a-b7a4-fbb12ff4db1c-client-ca-bundle\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.431887 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.431724 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/971cbd12-3157-444a-b7a4-fbb12ff4db1c-audit-log\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.532764 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.532726 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/971cbd12-3157-444a-b7a4-fbb12ff4db1c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.532939 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.532770 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/971cbd12-3157-444a-b7a4-fbb12ff4db1c-secret-metrics-server-client-certs\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.532939 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.532805 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/971cbd12-3157-444a-b7a4-fbb12ff4db1c-metrics-server-audit-profiles\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.532939 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.532841 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhffk\" (UniqueName: \"kubernetes.io/projected/971cbd12-3157-444a-b7a4-fbb12ff4db1c-kube-api-access-rhffk\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.532939 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.532875 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/971cbd12-3157-444a-b7a4-fbb12ff4db1c-secret-metrics-server-tls\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.532939 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.532916 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/971cbd12-3157-444a-b7a4-fbb12ff4db1c-client-ca-bundle\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.532939 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.532939 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/971cbd12-3157-444a-b7a4-fbb12ff4db1c-audit-log\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.533309 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.533282 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/971cbd12-3157-444a-b7a4-fbb12ff4db1c-audit-log\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.533510 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.533481 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/971cbd12-3157-444a-b7a4-fbb12ff4db1c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.534134 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.534111 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/971cbd12-3157-444a-b7a4-fbb12ff4db1c-metrics-server-audit-profiles\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.535879 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.535850 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/971cbd12-3157-444a-b7a4-fbb12ff4db1c-secret-metrics-server-tls\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.536001 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.535975 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/971cbd12-3157-444a-b7a4-fbb12ff4db1c-secret-metrics-server-client-certs\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.536097 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.536031 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/971cbd12-3157-444a-b7a4-fbb12ff4db1c-client-ca-bundle\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.541528 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.541505 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhffk\" (UniqueName: \"kubernetes.io/projected/971cbd12-3157-444a-b7a4-fbb12ff4db1c-kube-api-access-rhffk\") pod \"metrics-server-664cc87cf7-glvjn\" (UID: \"971cbd12-3157-444a-b7a4-fbb12ff4db1c\") " pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.670064 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.669988 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:22.698751 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.698719 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bjmwc"] Jul 18 01:49:22.718312 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.718277 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bjmwc" Jul 18 01:49:22.718985 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.718963 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bjmwc"] Jul 18 01:49:22.720829 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.720792 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 18 01:49:22.721437 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.721380 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-rgmpl\"" Jul 18 01:49:22.835818 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.835781 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwggb\" (UniqueName: \"kubernetes.io/projected/3c0d1cfd-e2ad-4ba2-9bc1-e773278b161e-kube-api-access-gwggb\") pod \"monitoring-plugin-7b7b7bc8bf-bjmwc\" (UID: \"3c0d1cfd-e2ad-4ba2-9bc1-e773278b161e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bjmwc" Jul 18 01:49:22.836013 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.835900 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3c0d1cfd-e2ad-4ba2-9bc1-e773278b161e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-bjmwc\" (UID: \"3c0d1cfd-e2ad-4ba2-9bc1-e773278b161e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bjmwc" Jul 18 01:49:22.936621 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.936539 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3c0d1cfd-e2ad-4ba2-9bc1-e773278b161e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-bjmwc\" (UID: \"3c0d1cfd-e2ad-4ba2-9bc1-e773278b161e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bjmwc" Jul 18 01:49:22.936621 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.936606 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwggb\" (UniqueName: \"kubernetes.io/projected/3c0d1cfd-e2ad-4ba2-9bc1-e773278b161e-kube-api-access-gwggb\") pod \"monitoring-plugin-7b7b7bc8bf-bjmwc\" (UID: \"3c0d1cfd-e2ad-4ba2-9bc1-e773278b161e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bjmwc" Jul 18 01:49:22.939255 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.939230 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3c0d1cfd-e2ad-4ba2-9bc1-e773278b161e-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-bjmwc\" (UID: \"3c0d1cfd-e2ad-4ba2-9bc1-e773278b161e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bjmwc" Jul 18 01:49:22.945953 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:22.945930 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwggb\" (UniqueName: \"kubernetes.io/projected/3c0d1cfd-e2ad-4ba2-9bc1-e773278b161e-kube-api-access-gwggb\") pod \"monitoring-plugin-7b7b7bc8bf-bjmwc\" (UID: \"3c0d1cfd-e2ad-4ba2-9bc1-e773278b161e\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bjmwc" Jul 18 01:49:23.030266 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:23.030230 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bjmwc" Jul 18 01:49:23.755822 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:49:23.755767 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31d64b4a_8386_4a23_be41_0df94f6cabf4.slice/crio-adaca32f98f0eeba41e7ad5b49d806d7840d817a5c200f584d9795fb8ff5455d WatchSource:0}: Error finding container adaca32f98f0eeba41e7ad5b49d806d7840d817a5c200f584d9795fb8ff5455d: Status 404 returned error can't find the container with id adaca32f98f0eeba41e7ad5b49d806d7840d817a5c200f584d9795fb8ff5455d Jul 18 01:49:23.916297 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:23.915532 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bzrjc"] Jul 18 01:49:23.933312 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:23.933211 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bjmwc"] Jul 18 01:49:23.936041 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:49:23.936015 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c0d1cfd_e2ad_4ba2_9bc1_e773278b161e.slice/crio-e9cc3ab8b8f8234a4ec63bbe14f07f05aa2d8b2edbdfb2002a9cd872fafccd62 WatchSource:0}: Error finding container e9cc3ab8b8f8234a4ec63bbe14f07f05aa2d8b2edbdfb2002a9cd872fafccd62: Status 404 returned error can't find the container with id e9cc3ab8b8f8234a4ec63bbe14f07f05aa2d8b2edbdfb2002a9cd872fafccd62 Jul 18 01:49:24.158186 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:24.158138 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-664cc87cf7-glvjn"] Jul 18 01:49:24.161224 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:49:24.161187 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod971cbd12_3157_444a_b7a4_fbb12ff4db1c.slice/crio-e7270f434d3bf36f02992199468f257190bab1063ef1fa1ff6e900dffd9abb78 WatchSource:0}: Error finding container e7270f434d3bf36f02992199468f257190bab1063ef1fa1ff6e900dffd9abb78: Status 404 returned error can't find the container with id e7270f434d3bf36f02992199468f257190bab1063ef1fa1ff6e900dffd9abb78 Jul 18 01:49:24.163522 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:24.163497 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5847499567-xn4cp"] Jul 18 01:49:24.167370 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:49:24.167332 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod632a8dc2_b00b_419a_a1f7_416a4cfa380c.slice/crio-92be7ecfd1a17d77390def1a32d7e7cc8877f374900904ab4963d8d5206429b5 WatchSource:0}: Error finding container 92be7ecfd1a17d77390def1a32d7e7cc8877f374900904ab4963d8d5206429b5: Status 404 returned error can't find the container with id 92be7ecfd1a17d77390def1a32d7e7cc8877f374900904ab4963d8d5206429b5 Jul 18 01:49:24.527074 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:24.526979 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c68mx" event={"ID":"31d64b4a-8386-4a23-be41-0df94f6cabf4","Type":"ContainerStarted","Data":"adaca32f98f0eeba41e7ad5b49d806d7840d817a5c200f584d9795fb8ff5455d"} Jul 18 01:49:24.532672 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:24.529888 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-kxctv" event={"ID":"41d8c727-0f8f-4fdf-ae90-c10fd2caaaff","Type":"ContainerStarted","Data":"a12765c3b12959be6922523e5c0cd1cb2548f6a2dc0ee42674adf0cb2f48c52b"} Jul 18 01:49:24.532672 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:24.531110 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-kxctv" Jul 18 01:49:24.532962 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:24.532935 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bzrjc" event={"ID":"163a2e0f-86f1-4c36-ba43-6d097cbec349","Type":"ContainerStarted","Data":"7ff701221a908c20a310408d32b99671eea9b32331455af15ccae73f209b3641"} Jul 18 01:49:24.534805 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:24.534744 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-kxctv" Jul 18 01:49:24.535583 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:24.535027 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bjmwc" event={"ID":"3c0d1cfd-e2ad-4ba2-9bc1-e773278b161e","Type":"ContainerStarted","Data":"e9cc3ab8b8f8234a4ec63bbe14f07f05aa2d8b2edbdfb2002a9cd872fafccd62"} Jul 18 01:49:24.537122 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:24.537053 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" event={"ID":"971cbd12-3157-444a-b7a4-fbb12ff4db1c","Type":"ContainerStarted","Data":"e7270f434d3bf36f02992199468f257190bab1063ef1fa1ff6e900dffd9abb78"} Jul 18 01:49:24.539553 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:24.539383 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5847499567-xn4cp" event={"ID":"632a8dc2-b00b-419a-a1f7-416a4cfa380c","Type":"ContainerStarted","Data":"f5988ddcd7d7e518eedaf9d53bdedcd3bc51f97ee0ed2884699acb3b67da2b7e"} Jul 18 01:49:24.539553 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:24.539424 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5847499567-xn4cp" event={"ID":"632a8dc2-b00b-419a-a1f7-416a4cfa380c","Type":"ContainerStarted","Data":"92be7ecfd1a17d77390def1a32d7e7cc8877f374900904ab4963d8d5206429b5"} Jul 18 01:49:24.539740 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:24.539615 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:49:24.552285 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:24.552169 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-kxctv" podStartSLOduration=1.757118878 podStartE2EDuration="19.552149364s" totalImagesPullingTime="17.795030486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:24.550173364 +0000 UTC m=+104.227597480" watchObservedRunningTime="2026-07-18 01:49:24.552149364 +0000 UTC m=+104.229573476" Jul 18 01:49:24.595873 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:24.594607 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5847499567-xn4cp" podStartSLOduration=103.594561261 podStartE2EDuration="1m43.594561261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:47:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:24.593919562 +0000 UTC m=+104.271343676" watchObservedRunningTime="2026-07-18 01:49:24.594561261 +0000 UTC m=+104.271985373" Jul 18 01:49:26.552895 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:26.552860 2570 generic.go:356] "Generic (PLEG): container finished" podID="31d64b4a-8386-4a23-be41-0df94f6cabf4" containerID="ed0a23e87596f798a1da42722f33921be31587800c3bc70c117bb478df35839f" exitCode=0 Jul 18 01:49:26.553543 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:26.552944 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c68mx" event={"ID":"31d64b4a-8386-4a23-be41-0df94f6cabf4","Type":"ContainerDied","Data":"ed0a23e87596f798a1da42722f33921be31587800c3bc70c117bb478df35839f"} Jul 18 01:49:28.563255 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:28.563215 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bjmwc" event={"ID":"3c0d1cfd-e2ad-4ba2-9bc1-e773278b161e","Type":"ContainerStarted","Data":"8d961b077290fde5c47dee81217c43a490406cc5ef03a176500716dbfe968329"} Jul 18 01:49:28.563751 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:28.563620 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bjmwc" Jul 18 01:49:28.565964 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:28.565924 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" event={"ID":"971cbd12-3157-444a-b7a4-fbb12ff4db1c","Type":"ContainerStarted","Data":"c8f56d290b0617c4b193cb0c4c17e44ef46d1ae0baf1880efa770b58aba96d4b"} Jul 18 01:49:28.569817 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:28.569212 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c68mx" event={"ID":"31d64b4a-8386-4a23-be41-0df94f6cabf4","Type":"ContainerStarted","Data":"17f327f3c291cf5f66d67fddb4414802e7754b21bf5e5e473320072d82155e5f"} Jul 18 01:49:28.569817 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:28.569268 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-c68mx" event={"ID":"31d64b4a-8386-4a23-be41-0df94f6cabf4","Type":"ContainerStarted","Data":"765fe052d656a5a820048604f98b67698822f27b4db51c3d9413d0492344ddfb"} Jul 18 01:49:28.569817 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:28.569325 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bjmwc" Jul 18 01:49:28.571491 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:28.571463 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bzrjc" event={"ID":"163a2e0f-86f1-4c36-ba43-6d097cbec349","Type":"ContainerStarted","Data":"94505a5b6f3bd49a1bc6f96c58ec390c3733ac07e4a85d0e3886a9d4af449865"} Jul 18 01:49:28.571596 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:28.571513 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bzrjc" event={"ID":"163a2e0f-86f1-4c36-ba43-6d097cbec349","Type":"ContainerStarted","Data":"05215fdab24c16af1d3392e981f246778b67685714f917cd49ecb5a82158bcd2"} Jul 18 01:49:28.571596 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:28.571561 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bzrjc" Jul 18 01:49:28.579287 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:28.579244 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-bjmwc" podStartSLOduration=2.634236557 podStartE2EDuration="6.579228358s" totalImagesPullingTime="3.944991801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:28.577680509 +0000 UTC m=+108.255104620" watchObservedRunningTime="2026-07-18 01:49:28.579228358 +0000 UTC m=+108.256652472" Jul 18 01:49:28.613878 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:28.613817 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" podStartSLOduration=2.894953633 podStartE2EDuration="6.613802917s" totalImagesPullingTime="3.718849284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:28.612395011 +0000 UTC m=+108.289819123" watchObservedRunningTime="2026-07-18 01:49:28.613802917 +0000 UTC m=+108.291227027" Jul 18 01:49:28.629600 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:28.629482 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bzrjc" podStartSLOduration=69.663293598 podStartE2EDuration="1m13.629464455s" totalImagesPullingTime="3.966170857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:28.628807892 +0000 UTC m=+108.306232004" watchObservedRunningTime="2026-07-18 01:49:28.629464455 +0000 UTC m=+108.306888565" Jul 18 01:49:28.649817 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:28.649753 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-c68mx" podStartSLOduration=9.722827133 podStartE2EDuration="11.649732304s" totalImagesPullingTime="1.827782206s" totalInitContainerRuntime="99.122965ms" isStatefulPod=true podCreationTimestamp="2026-07-18 01:49:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:28.647796563 +0000 UTC m=+108.325220697" watchObservedRunningTime="2026-07-18 01:49:28.649732304 +0000 UTC m=+108.327156415" Jul 18 01:49:29.276381 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:29.276343 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5847499567-xn4cp"] Jul 18 01:49:32.824320 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:32.824283 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-877fbb559-fqb9m"] Jul 18 01:49:32.848035 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:32.848003 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-877fbb559-fqb9m"] Jul 18 01:49:32.848175 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:32.848122 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:32.852224 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:32.852198 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 18 01:49:32.852365 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:32.852308 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 18 01:49:32.853497 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:32.853471 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 18 01:49:32.853611 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:32.853515 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 18 01:49:32.853611 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:32.853534 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 18 01:49:32.853611 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:32.853604 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qcq5b\"" Jul 18 01:49:32.856679 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:32.856661 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 18 01:49:32.929490 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:32.929454 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-oauth-config\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:32.929681 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:32.929549 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-oauth-serving-cert\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:32.929681 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:32.929582 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-service-ca\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:32.929823 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:32.929786 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7644\" (UniqueName: \"kubernetes.io/projected/1865d60f-1cc1-4915-aeb7-f5000a38332f-kube-api-access-q7644\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:32.929976 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:32.929832 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-serving-cert\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:32.929976 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:32.929869 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-config\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:32.929976 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:32.929892 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-trusted-ca-bundle\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:33.030513 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:33.030476 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-oauth-config\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:33.030682 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:33.030528 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-oauth-serving-cert\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:33.030682 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:33.030548 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-service-ca\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:33.030682 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:33.030578 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7644\" (UniqueName: \"kubernetes.io/projected/1865d60f-1cc1-4915-aeb7-f5000a38332f-kube-api-access-q7644\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:33.030682 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:33.030597 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-serving-cert\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:33.030682 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:33.030615 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-config\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:33.030682 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:33.030634 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-trusted-ca-bundle\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:33.031350 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:33.031325 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-service-ca\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:33.031467 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:33.031346 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-oauth-serving-cert\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:33.031467 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:33.031431 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-config\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:33.031585 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:33.031527 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-trusted-ca-bundle\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:33.033009 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:33.032984 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-oauth-config\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:33.033104 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:33.033069 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-serving-cert\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:33.039189 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:33.039168 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7644\" (UniqueName: \"kubernetes.io/projected/1865d60f-1cc1-4915-aeb7-f5000a38332f-kube-api-access-q7644\") pod \"console-877fbb559-fqb9m\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:33.158588 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:33.158504 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:33.314432 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:33.314406 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-877fbb559-fqb9m"] Jul 18 01:49:33.316621 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:49:33.316595 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1865d60f_1cc1_4915_aeb7_f5000a38332f.slice/crio-36ddf82106ca8800064d96d5dfa9948353aeeae6d150479354cb2e207acb59b3 WatchSource:0}: Error finding container 36ddf82106ca8800064d96d5dfa9948353aeeae6d150479354cb2e207acb59b3: Status 404 returned error can't find the container with id 36ddf82106ca8800064d96d5dfa9948353aeeae6d150479354cb2e207acb59b3 Jul 18 01:49:33.593096 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:33.593066 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-877fbb559-fqb9m" event={"ID":"1865d60f-1cc1-4915-aeb7-f5000a38332f","Type":"ContainerStarted","Data":"36ddf82106ca8800064d96d5dfa9948353aeeae6d150479354cb2e207acb59b3"} Jul 18 01:49:36.606590 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:36.606508 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-877fbb559-fqb9m" event={"ID":"1865d60f-1cc1-4915-aeb7-f5000a38332f","Type":"ContainerStarted","Data":"394a696198171a7f0600485373b6e8cd14582744d6e0e88aa154a049fd48273f"} Jul 18 01:49:36.625310 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:36.625263 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-877fbb559-fqb9m" podStartSLOduration=1.639012567 podStartE2EDuration="4.625248345s" totalImagesPullingTime="2.986235778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:49:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:36.623381465 +0000 UTC m=+116.300805577" watchObservedRunningTime="2026-07-18 01:49:36.625248345 +0000 UTC m=+116.302672455" Jul 18 01:49:38.577417 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:38.577384 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bzrjc" Jul 18 01:49:39.281926 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:39.281897 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:49:42.670407 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:42.670375 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:42.670877 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:42.670436 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:49:43.159460 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:43.159425 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:43.159664 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:43.159487 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:43.164231 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:43.164207 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:43.636096 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:43.636069 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:49:49.785540 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:49.785499 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs\") pod \"network-metrics-daemon-5nqdq\" (UID: \"bf7391f7-3b4a-4eb5-ab3a-b540543d4907\") " pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:49:49.787779 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:49.787759 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf7391f7-3b4a-4eb5-ab3a-b540543d4907-metrics-certs\") pod \"network-metrics-daemon-5nqdq\" (UID: \"bf7391f7-3b4a-4eb5-ab3a-b540543d4907\") " pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:49:49.901623 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:49.901589 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tqgxz\"" Jul 18 01:49:49.909794 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:49.909769 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5nqdq" Jul 18 01:49:50.225323 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:50.225286 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5nqdq"] Jul 18 01:49:50.228285 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:49:50.228257 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf7391f7_3b4a_4eb5_ab3a_b540543d4907.slice/crio-b47356435dd77d847d4186e50c5e1c073f583eaf8b9f94a9d4eef63ce3f572c3 WatchSource:0}: Error finding container b47356435dd77d847d4186e50c5e1c073f583eaf8b9f94a9d4eef63ce3f572c3: Status 404 returned error can't find the container with id b47356435dd77d847d4186e50c5e1c073f583eaf8b9f94a9d4eef63ce3f572c3 Jul 18 01:49:50.659942 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:50.659907 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5nqdq" event={"ID":"bf7391f7-3b4a-4eb5-ab3a-b540543d4907","Type":"ContainerStarted","Data":"b47356435dd77d847d4186e50c5e1c073f583eaf8b9f94a9d4eef63ce3f572c3"} Jul 18 01:49:52.668950 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:52.668909 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5nqdq" event={"ID":"bf7391f7-3b4a-4eb5-ab3a-b540543d4907","Type":"ContainerStarted","Data":"5f11248c004c0982331ea548e3bb3ebd59039b6688b79f05a0574032a8f7eb33"} Jul 18 01:49:52.669422 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:52.668968 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5nqdq" event={"ID":"bf7391f7-3b4a-4eb5-ab3a-b540543d4907","Type":"ContainerStarted","Data":"b0abf061f217e6734f45261d7df8d3c27337ebf0221499ea44f9bbe21ef6f36f"} Jul 18 01:49:52.687395 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:52.687347 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5nqdq" podStartSLOduration=130.278083488 podStartE2EDuration="2m11.687331497s" totalImagesPullingTime="1.409248009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:47:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:52.68677386 +0000 UTC m=+132.364197976" watchObservedRunningTime="2026-07-18 01:49:52.687331497 +0000 UTC m=+132.364755608" Jul 18 01:49:54.300460 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.300406 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5847499567-xn4cp" podUID="632a8dc2-b00b-419a-a1f7-416a4cfa380c" containerName="registry" containerID="cri-o://f5988ddcd7d7e518eedaf9d53bdedcd3bc51f97ee0ed2884699acb3b67da2b7e" gracePeriod=30 Jul 18 01:49:54.539098 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.539074 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:49:54.677416 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.677324 2570 generic.go:356] "Generic (PLEG): container finished" podID="632a8dc2-b00b-419a-a1f7-416a4cfa380c" containerID="f5988ddcd7d7e518eedaf9d53bdedcd3bc51f97ee0ed2884699acb3b67da2b7e" exitCode=0 Jul 18 01:49:54.677416 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.677373 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5847499567-xn4cp" Jul 18 01:49:54.677416 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.677383 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5847499567-xn4cp" event={"ID":"632a8dc2-b00b-419a-a1f7-416a4cfa380c","Type":"ContainerDied","Data":"f5988ddcd7d7e518eedaf9d53bdedcd3bc51f97ee0ed2884699acb3b67da2b7e"} Jul 18 01:49:54.677623 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.677434 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5847499567-xn4cp" event={"ID":"632a8dc2-b00b-419a-a1f7-416a4cfa380c","Type":"ContainerDied","Data":"92be7ecfd1a17d77390def1a32d7e7cc8877f374900904ab4963d8d5206429b5"} Jul 18 01:49:54.677623 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.677461 2570 scope.go:117] "RemoveContainer" containerID="f5988ddcd7d7e518eedaf9d53bdedcd3bc51f97ee0ed2884699acb3b67da2b7e" Jul 18 01:49:54.686564 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.686545 2570 scope.go:117] "RemoveContainer" containerID="f5988ddcd7d7e518eedaf9d53bdedcd3bc51f97ee0ed2884699acb3b67da2b7e" Jul 18 01:49:54.687024 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:49:54.687000 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5988ddcd7d7e518eedaf9d53bdedcd3bc51f97ee0ed2884699acb3b67da2b7e\": container with ID starting with f5988ddcd7d7e518eedaf9d53bdedcd3bc51f97ee0ed2884699acb3b67da2b7e not found: ID does not exist" containerID="f5988ddcd7d7e518eedaf9d53bdedcd3bc51f97ee0ed2884699acb3b67da2b7e" Jul 18 01:49:54.687101 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.687033 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5988ddcd7d7e518eedaf9d53bdedcd3bc51f97ee0ed2884699acb3b67da2b7e"} err="failed to get container status \"f5988ddcd7d7e518eedaf9d53bdedcd3bc51f97ee0ed2884699acb3b67da2b7e\": rpc error: code = NotFound desc = could not find container \"f5988ddcd7d7e518eedaf9d53bdedcd3bc51f97ee0ed2884699acb3b67da2b7e\": container with ID starting with f5988ddcd7d7e518eedaf9d53bdedcd3bc51f97ee0ed2884699acb3b67da2b7e not found: ID does not exist" Jul 18 01:49:54.729633 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.729608 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-bound-sa-token\") pod \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " Jul 18 01:49:54.729736 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.729636 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cm8zc\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-kube-api-access-cm8zc\") pod \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " Jul 18 01:49:54.729736 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.729683 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/632a8dc2-b00b-419a-a1f7-416a4cfa380c-trusted-ca\") pod \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " Jul 18 01:49:54.729736 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.729720 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/632a8dc2-b00b-419a-a1f7-416a4cfa380c-image-registry-private-configuration\") pod \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " Jul 18 01:49:54.729838 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.729739 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/632a8dc2-b00b-419a-a1f7-416a4cfa380c-ca-trust-extracted\") pod \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " Jul 18 01:49:54.729838 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.729762 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/632a8dc2-b00b-419a-a1f7-416a4cfa380c-installation-pull-secrets\") pod \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " Jul 18 01:49:54.729838 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.729810 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-certificates\") pod \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " Jul 18 01:49:54.729961 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.729852 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls\") pod \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\" (UID: \"632a8dc2-b00b-419a-a1f7-416a4cfa380c\") " Jul 18 01:49:54.731934 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.730492 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/632a8dc2-b00b-419a-a1f7-416a4cfa380c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "632a8dc2-b00b-419a-a1f7-416a4cfa380c" (UID: "632a8dc2-b00b-419a-a1f7-416a4cfa380c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:49:54.732806 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.732609 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "632a8dc2-b00b-419a-a1f7-416a4cfa380c" (UID: "632a8dc2-b00b-419a-a1f7-416a4cfa380c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:49:54.732959 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.732897 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "632a8dc2-b00b-419a-a1f7-416a4cfa380c" (UID: "632a8dc2-b00b-419a-a1f7-416a4cfa380c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:49:54.733048 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.733008 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "632a8dc2-b00b-419a-a1f7-416a4cfa380c" (UID: "632a8dc2-b00b-419a-a1f7-416a4cfa380c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:49:54.733154 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.733114 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/632a8dc2-b00b-419a-a1f7-416a4cfa380c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "632a8dc2-b00b-419a-a1f7-416a4cfa380c" (UID: "632a8dc2-b00b-419a-a1f7-416a4cfa380c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:49:54.734699 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.733338 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-kube-api-access-cm8zc" (OuterVolumeSpecName: "kube-api-access-cm8zc") pod "632a8dc2-b00b-419a-a1f7-416a4cfa380c" (UID: "632a8dc2-b00b-419a-a1f7-416a4cfa380c"). InnerVolumeSpecName "kube-api-access-cm8zc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:49:54.737391 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.737178 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/632a8dc2-b00b-419a-a1f7-416a4cfa380c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "632a8dc2-b00b-419a-a1f7-416a4cfa380c" (UID: "632a8dc2-b00b-419a-a1f7-416a4cfa380c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:49:54.740607 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.740583 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/632a8dc2-b00b-419a-a1f7-416a4cfa380c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "632a8dc2-b00b-419a-a1f7-416a4cfa380c" (UID: "632a8dc2-b00b-419a-a1f7-416a4cfa380c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:49:54.831511 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.831475 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-tls\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:49:54.831511 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.831503 2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-bound-sa-token\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:49:54.831511 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.831512 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cm8zc\" (UniqueName: \"kubernetes.io/projected/632a8dc2-b00b-419a-a1f7-416a4cfa380c-kube-api-access-cm8zc\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:49:54.831761 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.831522 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/632a8dc2-b00b-419a-a1f7-416a4cfa380c-trusted-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:49:54.831761 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.831532 2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/632a8dc2-b00b-419a-a1f7-416a4cfa380c-image-registry-private-configuration\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:49:54.831761 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.831541 2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/632a8dc2-b00b-419a-a1f7-416a4cfa380c-ca-trust-extracted\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:49:54.831761 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.831549 2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/632a8dc2-b00b-419a-a1f7-416a4cfa380c-installation-pull-secrets\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:49:54.831761 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.831557 2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/632a8dc2-b00b-419a-a1f7-416a4cfa380c-registry-certificates\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:49:54.992427 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.992337 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5847499567-xn4cp"] Jul 18 01:49:54.996095 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:54.996068 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5847499567-xn4cp"] Jul 18 01:49:56.892995 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:49:56.892962 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="632a8dc2-b00b-419a-a1f7-416a4cfa380c" path="/var/lib/kubelet/pods/632a8dc2-b00b-419a-a1f7-416a4cfa380c/volumes" Jul 18 01:50:02.676064 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:02.676031 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:50:02.679886 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:02.679858 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-664cc87cf7-glvjn" Jul 18 01:50:09.736261 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:09.736180 2570 generic.go:356] "Generic (PLEG): container finished" podID="eecae00a-6435-4154-9386-988f7365b0cb" containerID="8f77beadf161857885f6701369760e27e39a457b67c4d19f4f20623260efc63c" exitCode=0 Jul 18 01:50:09.736617 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:09.736253 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn" event={"ID":"eecae00a-6435-4154-9386-988f7365b0cb","Type":"ContainerDied","Data":"8f77beadf161857885f6701369760e27e39a457b67c4d19f4f20623260efc63c"} Jul 18 01:50:09.736682 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:09.736627 2570 scope.go:117] "RemoveContainer" containerID="8f77beadf161857885f6701369760e27e39a457b67c4d19f4f20623260efc63c" Jul 18 01:50:10.741289 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:10.741254 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-czfpn" event={"ID":"eecae00a-6435-4154-9386-988f7365b0cb","Type":"ContainerStarted","Data":"9cc30e5dcda4ffc7b4a2b01987560c472835783dd38c130865039eb236c26ba9"} Jul 18 01:50:37.727808 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.727733 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b55dbc779-c9wqq"] Jul 18 01:50:37.728223 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.728092 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="632a8dc2-b00b-419a-a1f7-416a4cfa380c" containerName="registry" Jul 18 01:50:37.728223 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.728106 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="632a8dc2-b00b-419a-a1f7-416a4cfa380c" containerName="registry" Jul 18 01:50:37.728223 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.728182 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="632a8dc2-b00b-419a-a1f7-416a4cfa380c" containerName="registry" Jul 18 01:50:37.734589 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.734369 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.740827 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.740799 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b55dbc779-c9wqq"] Jul 18 01:50:37.866664 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.866616 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-oauth-config\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.866852 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.866673 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-service-ca\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.866852 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.866721 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-oauth-serving-cert\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.866852 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.866740 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws6mj\" (UniqueName: \"kubernetes.io/projected/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-kube-api-access-ws6mj\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.866852 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.866766 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-trusted-ca-bundle\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.866852 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.866801 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-serving-cert\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.866852 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.866824 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-config\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.967615 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.967581 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-service-ca\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.967615 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.967616 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-oauth-serving-cert\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.967885 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.967637 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws6mj\" (UniqueName: \"kubernetes.io/projected/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-kube-api-access-ws6mj\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.967885 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.967767 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-trusted-ca-bundle\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.967885 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.967826 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-serving-cert\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.967885 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.967856 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-config\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.968092 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.968003 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-oauth-config\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.968481 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.968457 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-service-ca\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.968637 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.968616 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-oauth-serving-cert\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.968742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.968616 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-config\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.968742 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.968695 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-trusted-ca-bundle\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.970400 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.970374 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-oauth-config\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.970498 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.970470 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-serving-cert\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:37.976160 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:37.976138 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws6mj\" (UniqueName: \"kubernetes.io/projected/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-kube-api-access-ws6mj\") pod \"console-6b55dbc779-c9wqq\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:38.046622 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:38.046591 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:38.171477 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:38.171440 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b55dbc779-c9wqq"] Jul 18 01:50:38.175190 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:50:38.175159 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a86969b_dcb5_4cf7_8007_dd9ca721b6df.slice/crio-fe641a2ef41ef4755d91aad1c797383f127513d082d3c4cef2d17680c668321f WatchSource:0}: Error finding container fe641a2ef41ef4755d91aad1c797383f127513d082d3c4cef2d17680c668321f: Status 404 returned error can't find the container with id fe641a2ef41ef4755d91aad1c797383f127513d082d3c4cef2d17680c668321f Jul 18 01:50:38.845725 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:38.845686 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b55dbc779-c9wqq" event={"ID":"9a86969b-dcb5-4cf7-8007-dd9ca721b6df","Type":"ContainerStarted","Data":"958b997d23b43a03c82115ca1e4aee6e661d0a449c5a6f92e305ea5cff93844d"} Jul 18 01:50:38.846097 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:38.845741 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b55dbc779-c9wqq" event={"ID":"9a86969b-dcb5-4cf7-8007-dd9ca721b6df","Type":"ContainerStarted","Data":"fe641a2ef41ef4755d91aad1c797383f127513d082d3c4cef2d17680c668321f"} Jul 18 01:50:38.862727 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:38.862675 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b55dbc779-c9wqq" podStartSLOduration=1.8626599860000002 podStartE2EDuration="1.862659986s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:50:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:50:38.860928028 +0000 UTC m=+178.538352150" watchObservedRunningTime="2026-07-18 01:50:38.862659986 +0000 UTC m=+178.540084091" Jul 18 01:50:48.046984 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:48.046947 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:48.047460 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:48.047035 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:48.051588 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:48.051566 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:48.887616 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:48.887589 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:50:48.931971 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:50:48.931940 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-877fbb559-fqb9m"] Jul 18 01:51:08.776459 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.776420 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-859b94ff96-b8rg9"] Jul 18 01:51:08.779325 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.779309 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-859b94ff96-b8rg9" Jul 18 01:51:08.781740 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.781713 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 18 01:51:08.782734 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.782712 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-lpngg\"" Jul 18 01:51:08.782843 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.782815 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 18 01:51:08.782907 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.782846 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 18 01:51:08.782948 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.782928 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 18 01:51:08.786723 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.786701 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-859b94ff96-b8rg9"] Jul 18 01:51:08.812427 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.812390 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nwcl\" (UniqueName: \"kubernetes.io/projected/dda7ef16-3f73-439e-8993-95e7fb4412fc-kube-api-access-9nwcl\") pod \"managed-serviceaccount-addon-agent-859b94ff96-b8rg9\" (UID: \"dda7ef16-3f73-439e-8993-95e7fb4412fc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-859b94ff96-b8rg9" Jul 18 01:51:08.812573 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.812437 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dda7ef16-3f73-439e-8993-95e7fb4412fc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-859b94ff96-b8rg9\" (UID: \"dda7ef16-3f73-439e-8993-95e7fb4412fc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-859b94ff96-b8rg9" Jul 18 01:51:08.851165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.851137 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr"] Jul 18 01:51:08.854373 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.854358 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:08.856475 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.856452 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 18 01:51:08.856691 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.856473 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 18 01:51:08.856781 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.856750 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 18 01:51:08.856781 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.856767 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 18 01:51:08.864927 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.864902 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr"] Jul 18 01:51:08.913478 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.913448 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8eb70b2-047a-4994-a47d-7361732a2afe-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:08.913618 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.913488 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dda7ef16-3f73-439e-8993-95e7fb4412fc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-859b94ff96-b8rg9\" (UID: \"dda7ef16-3f73-439e-8993-95e7fb4412fc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-859b94ff96-b8rg9" Jul 18 01:51:08.913618 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.913513 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b8eb70b2-047a-4994-a47d-7361732a2afe-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:08.913618 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.913569 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfjpm\" (UniqueName: \"kubernetes.io/projected/b8eb70b2-047a-4994-a47d-7361732a2afe-kube-api-access-pfjpm\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:08.913618 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.913598 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8eb70b2-047a-4994-a47d-7361732a2afe-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:08.913804 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.913632 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b8eb70b2-047a-4994-a47d-7361732a2afe-hub\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:08.913804 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.913676 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b8eb70b2-047a-4994-a47d-7361732a2afe-ca\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:08.913804 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.913711 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nwcl\" (UniqueName: \"kubernetes.io/projected/dda7ef16-3f73-439e-8993-95e7fb4412fc-kube-api-access-9nwcl\") pod \"managed-serviceaccount-addon-agent-859b94ff96-b8rg9\" (UID: \"dda7ef16-3f73-439e-8993-95e7fb4412fc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-859b94ff96-b8rg9" Jul 18 01:51:08.915888 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.915869 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dda7ef16-3f73-439e-8993-95e7fb4412fc-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-859b94ff96-b8rg9\" (UID: \"dda7ef16-3f73-439e-8993-95e7fb4412fc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-859b94ff96-b8rg9" Jul 18 01:51:08.921181 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:08.921153 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nwcl\" (UniqueName: \"kubernetes.io/projected/dda7ef16-3f73-439e-8993-95e7fb4412fc-kube-api-access-9nwcl\") pod \"managed-serviceaccount-addon-agent-859b94ff96-b8rg9\" (UID: \"dda7ef16-3f73-439e-8993-95e7fb4412fc\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-859b94ff96-b8rg9" Jul 18 01:51:09.014534 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.014495 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b8eb70b2-047a-4994-a47d-7361732a2afe-hub\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:09.014534 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.014532 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b8eb70b2-047a-4994-a47d-7361732a2afe-ca\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:09.014776 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.014572 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8eb70b2-047a-4994-a47d-7361732a2afe-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:09.014776 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.014601 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b8eb70b2-047a-4994-a47d-7361732a2afe-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:09.014776 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.014677 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfjpm\" (UniqueName: \"kubernetes.io/projected/b8eb70b2-047a-4994-a47d-7361732a2afe-kube-api-access-pfjpm\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:09.014776 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.014739 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8eb70b2-047a-4994-a47d-7361732a2afe-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:09.015391 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.015344 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8eb70b2-047a-4994-a47d-7361732a2afe-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:09.017227 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.017199 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/b8eb70b2-047a-4994-a47d-7361732a2afe-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:09.017318 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.017201 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b8eb70b2-047a-4994-a47d-7361732a2afe-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:09.017477 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.017456 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/b8eb70b2-047a-4994-a47d-7361732a2afe-ca\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:09.017513 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.017502 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/b8eb70b2-047a-4994-a47d-7361732a2afe-hub\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:09.022208 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.022187 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfjpm\" (UniqueName: \"kubernetes.io/projected/b8eb70b2-047a-4994-a47d-7361732a2afe-kube-api-access-pfjpm\") pod \"cluster-proxy-proxy-agent-7dc6c64f78-hz2qr\" (UID: \"b8eb70b2-047a-4994-a47d-7361732a2afe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:09.093948 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.093924 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-859b94ff96-b8rg9" Jul 18 01:51:09.162866 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.162835 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" Jul 18 01:51:09.221973 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.221938 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-859b94ff96-b8rg9"] Jul 18 01:51:09.224299 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:51:09.224271 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddda7ef16_3f73_439e_8993_95e7fb4412fc.slice/crio-a8b1131eee3b7970f688bb68595c5d5f5ceca9adb05bd28bca48ca9db2b66c1c WatchSource:0}: Error finding container a8b1131eee3b7970f688bb68595c5d5f5ceca9adb05bd28bca48ca9db2b66c1c: Status 404 returned error can't find the container with id a8b1131eee3b7970f688bb68595c5d5f5ceca9adb05bd28bca48ca9db2b66c1c Jul 18 01:51:09.295299 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.295277 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr"] Jul 18 01:51:09.297710 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:51:09.297633 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8eb70b2_047a_4994_a47d_7361732a2afe.slice/crio-62e06e91cab0662ce6b69cc0fcc65590f5b8a9f209c4d61c9cb1622c66010198 WatchSource:0}: Error finding container 62e06e91cab0662ce6b69cc0fcc65590f5b8a9f209c4d61c9cb1622c66010198: Status 404 returned error can't find the container with id 62e06e91cab0662ce6b69cc0fcc65590f5b8a9f209c4d61c9cb1622c66010198 Jul 18 01:51:09.955879 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.955829 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" event={"ID":"b8eb70b2-047a-4994-a47d-7361732a2afe","Type":"ContainerStarted","Data":"62e06e91cab0662ce6b69cc0fcc65590f5b8a9f209c4d61c9cb1622c66010198"} Jul 18 01:51:09.956935 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:09.956914 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-859b94ff96-b8rg9" event={"ID":"dda7ef16-3f73-439e-8993-95e7fb4412fc","Type":"ContainerStarted","Data":"a8b1131eee3b7970f688bb68595c5d5f5ceca9adb05bd28bca48ca9db2b66c1c"} Jul 18 01:51:13.953456 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:13.953392 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-877fbb559-fqb9m" podUID="1865d60f-1cc1-4915-aeb7-f5000a38332f" containerName="console" containerID="cri-o://394a696198171a7f0600485373b6e8cd14582744d6e0e88aa154a049fd48273f" gracePeriod=15 Jul 18 01:51:13.977300 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:13.977261 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" event={"ID":"b8eb70b2-047a-4994-a47d-7361732a2afe","Type":"ContainerStarted","Data":"7c60b262284035feeab1107b5e961e4170f3428fda5a4b8364d371c828e0843c"} Jul 18 01:51:13.977455 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:13.977316 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" event={"ID":"b8eb70b2-047a-4994-a47d-7361732a2afe","Type":"ContainerStarted","Data":"2a826bd7e5c6b0f5901434afe9ccc355773dc816c5f0cbe6754cb84304e2e746"} Jul 18 01:51:13.977455 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:13.977335 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" event={"ID":"b8eb70b2-047a-4994-a47d-7361732a2afe","Type":"ContainerStarted","Data":"89e4f5f43233f9bdacbf628cd4cd8efd30f9d8a55ab7e359e630eeb6fe4d2354"} Jul 18 01:51:13.978611 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:13.978588 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-859b94ff96-b8rg9" event={"ID":"dda7ef16-3f73-439e-8993-95e7fb4412fc","Type":"ContainerStarted","Data":"56057d5e8cb19267f67d68017e1b38eaa2e8d0ad5047d6b59865a2a69efc0852"} Jul 18 01:51:13.993823 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:13.993768 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7dc6c64f78-hz2qr" podStartSLOduration=1.929438354 podStartE2EDuration="5.993751436s" totalImagesPullingTime="4.064313082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:51:13.993587399 +0000 UTC m=+213.671011514" watchObservedRunningTime="2026-07-18 01:51:13.993751436 +0000 UTC m=+213.671175548" Jul 18 01:51:14.007018 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.006970 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-859b94ff96-b8rg9" podStartSLOduration=1.880697491 podStartE2EDuration="6.006958109s" totalImagesPullingTime="4.126260618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:51:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:51:14.006840356 +0000 UTC m=+213.684264466" watchObservedRunningTime="2026-07-18 01:51:14.006958109 +0000 UTC m=+213.684382220" Jul 18 01:51:14.186825 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.186796 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:51:14.257597 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.257513 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-oauth-config\") pod \"1865d60f-1cc1-4915-aeb7-f5000a38332f\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " Jul 18 01:51:14.257597 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.257563 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-serving-cert\") pod \"1865d60f-1cc1-4915-aeb7-f5000a38332f\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " Jul 18 01:51:14.257597 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.257588 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-oauth-serving-cert\") pod \"1865d60f-1cc1-4915-aeb7-f5000a38332f\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " Jul 18 01:51:14.257862 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.257605 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7644\" (UniqueName: \"kubernetes.io/projected/1865d60f-1cc1-4915-aeb7-f5000a38332f-kube-api-access-q7644\") pod \"1865d60f-1cc1-4915-aeb7-f5000a38332f\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " Jul 18 01:51:14.257862 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.257663 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-service-ca\") pod \"1865d60f-1cc1-4915-aeb7-f5000a38332f\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " Jul 18 01:51:14.257862 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.257697 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-config\") pod \"1865d60f-1cc1-4915-aeb7-f5000a38332f\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " Jul 18 01:51:14.257862 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.257718 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-trusted-ca-bundle\") pod \"1865d60f-1cc1-4915-aeb7-f5000a38332f\" (UID: \"1865d60f-1cc1-4915-aeb7-f5000a38332f\") " Jul 18 01:51:14.258058 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.258007 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1865d60f-1cc1-4915-aeb7-f5000a38332f" (UID: "1865d60f-1cc1-4915-aeb7-f5000a38332f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:51:14.258169 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.258147 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-config" (OuterVolumeSpecName: "console-config") pod "1865d60f-1cc1-4915-aeb7-f5000a38332f" (UID: "1865d60f-1cc1-4915-aeb7-f5000a38332f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:51:14.258218 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.258164 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-service-ca" (OuterVolumeSpecName: "service-ca") pod "1865d60f-1cc1-4915-aeb7-f5000a38332f" (UID: "1865d60f-1cc1-4915-aeb7-f5000a38332f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:51:14.258661 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.258612 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1865d60f-1cc1-4915-aeb7-f5000a38332f" (UID: "1865d60f-1cc1-4915-aeb7-f5000a38332f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:51:14.260674 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.260436 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1865d60f-1cc1-4915-aeb7-f5000a38332f-kube-api-access-q7644" (OuterVolumeSpecName: "kube-api-access-q7644") pod "1865d60f-1cc1-4915-aeb7-f5000a38332f" (UID: "1865d60f-1cc1-4915-aeb7-f5000a38332f"). InnerVolumeSpecName "kube-api-access-q7644". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:51:14.261686 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.260823 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1865d60f-1cc1-4915-aeb7-f5000a38332f" (UID: "1865d60f-1cc1-4915-aeb7-f5000a38332f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:51:14.261686 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.261366 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1865d60f-1cc1-4915-aeb7-f5000a38332f" (UID: "1865d60f-1cc1-4915-aeb7-f5000a38332f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:51:14.358238 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.358200 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-service-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:51:14.358238 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.358232 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-config\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:51:14.358238 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.358243 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-trusted-ca-bundle\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:51:14.358464 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.358254 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-oauth-config\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:51:14.358464 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.358263 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1865d60f-1cc1-4915-aeb7-f5000a38332f-console-serving-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:51:14.358464 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.358270 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1865d60f-1cc1-4915-aeb7-f5000a38332f-oauth-serving-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:51:14.358464 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.358279 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7644\" (UniqueName: \"kubernetes.io/projected/1865d60f-1cc1-4915-aeb7-f5000a38332f-kube-api-access-q7644\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:51:14.983475 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.983443 2570 generic.go:356] "Generic (PLEG): container finished" podID="1865d60f-1cc1-4915-aeb7-f5000a38332f" containerID="394a696198171a7f0600485373b6e8cd14582744d6e0e88aa154a049fd48273f" exitCode=0 Jul 18 01:51:14.983888 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.983505 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-877fbb559-fqb9m" Jul 18 01:51:14.983888 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.983540 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-877fbb559-fqb9m" event={"ID":"1865d60f-1cc1-4915-aeb7-f5000a38332f","Type":"ContainerDied","Data":"394a696198171a7f0600485373b6e8cd14582744d6e0e88aa154a049fd48273f"} Jul 18 01:51:14.983888 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.983594 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-877fbb559-fqb9m" event={"ID":"1865d60f-1cc1-4915-aeb7-f5000a38332f","Type":"ContainerDied","Data":"36ddf82106ca8800064d96d5dfa9948353aeeae6d150479354cb2e207acb59b3"} Jul 18 01:51:14.983888 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.983619 2570 scope.go:117] "RemoveContainer" containerID="394a696198171a7f0600485373b6e8cd14582744d6e0e88aa154a049fd48273f" Jul 18 01:51:14.992010 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.991993 2570 scope.go:117] "RemoveContainer" containerID="394a696198171a7f0600485373b6e8cd14582744d6e0e88aa154a049fd48273f" Jul 18 01:51:14.992292 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:51:14.992273 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"394a696198171a7f0600485373b6e8cd14582744d6e0e88aa154a049fd48273f\": container with ID starting with 394a696198171a7f0600485373b6e8cd14582744d6e0e88aa154a049fd48273f not found: ID does not exist" containerID="394a696198171a7f0600485373b6e8cd14582744d6e0e88aa154a049fd48273f" Jul 18 01:51:14.992364 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.992302 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"394a696198171a7f0600485373b6e8cd14582744d6e0e88aa154a049fd48273f"} err="failed to get container status \"394a696198171a7f0600485373b6e8cd14582744d6e0e88aa154a049fd48273f\": rpc error: code = NotFound desc = could not find container \"394a696198171a7f0600485373b6e8cd14582744d6e0e88aa154a049fd48273f\": container with ID starting with 394a696198171a7f0600485373b6e8cd14582744d6e0e88aa154a049fd48273f not found: ID does not exist" Jul 18 01:51:14.999783 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:14.999748 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-877fbb559-fqb9m"] Jul 18 01:51:15.001888 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:15.001861 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-877fbb559-fqb9m"] Jul 18 01:51:16.892876 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:51:16.892842 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1865d60f-1cc1-4915-aeb7-f5000a38332f" path="/var/lib/kubelet/pods/1865d60f-1cc1-4915-aeb7-f5000a38332f/volumes" Jul 18 01:52:35.446154 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.446118 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv"] Jul 18 01:52:35.446565 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.446401 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1865d60f-1cc1-4915-aeb7-f5000a38332f" containerName="console" Jul 18 01:52:35.446565 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.446412 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1865d60f-1cc1-4915-aeb7-f5000a38332f" containerName="console" Jul 18 01:52:35.446565 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.446471 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1865d60f-1cc1-4915-aeb7-f5000a38332f" containerName="console" Jul 18 01:52:35.449391 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.449371 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:35.451989 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.451964 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 18 01:52:35.452122 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.451989 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 18 01:52:35.452122 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.451969 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 18 01:52:35.452122 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.452047 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8xwps\"" Jul 18 01:52:35.452304 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.452180 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 18 01:52:35.466241 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.466219 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv"] Jul 18 01:52:35.495045 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.495010 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a898a04a-8c9c-46f1-b225-f108ebd7fe1b-webhook-cert\") pod \"opendatahub-operator-controller-manager-6667d74d96-gc6zv\" (UID: \"a898a04a-8c9c-46f1-b225-f108ebd7fe1b\") " pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:35.495045 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.495049 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a898a04a-8c9c-46f1-b225-f108ebd7fe1b-manifests\") pod \"opendatahub-operator-controller-manager-6667d74d96-gc6zv\" (UID: \"a898a04a-8c9c-46f1-b225-f108ebd7fe1b\") " pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:35.495229 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.495078 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a898a04a-8c9c-46f1-b225-f108ebd7fe1b-tmp\") pod \"opendatahub-operator-controller-manager-6667d74d96-gc6zv\" (UID: \"a898a04a-8c9c-46f1-b225-f108ebd7fe1b\") " pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:35.495229 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.495095 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbmcj\" (UniqueName: \"kubernetes.io/projected/a898a04a-8c9c-46f1-b225-f108ebd7fe1b-kube-api-access-nbmcj\") pod \"opendatahub-operator-controller-manager-6667d74d96-gc6zv\" (UID: \"a898a04a-8c9c-46f1-b225-f108ebd7fe1b\") " pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:35.495229 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.495120 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a898a04a-8c9c-46f1-b225-f108ebd7fe1b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6667d74d96-gc6zv\" (UID: \"a898a04a-8c9c-46f1-b225-f108ebd7fe1b\") " pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:35.595874 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.595829 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a898a04a-8c9c-46f1-b225-f108ebd7fe1b-webhook-cert\") pod \"opendatahub-operator-controller-manager-6667d74d96-gc6zv\" (UID: \"a898a04a-8c9c-46f1-b225-f108ebd7fe1b\") " pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:35.596055 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.595887 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a898a04a-8c9c-46f1-b225-f108ebd7fe1b-manifests\") pod \"opendatahub-operator-controller-manager-6667d74d96-gc6zv\" (UID: \"a898a04a-8c9c-46f1-b225-f108ebd7fe1b\") " pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:35.596055 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.595947 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a898a04a-8c9c-46f1-b225-f108ebd7fe1b-tmp\") pod \"opendatahub-operator-controller-manager-6667d74d96-gc6zv\" (UID: \"a898a04a-8c9c-46f1-b225-f108ebd7fe1b\") " pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:35.596055 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.595971 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbmcj\" (UniqueName: \"kubernetes.io/projected/a898a04a-8c9c-46f1-b225-f108ebd7fe1b-kube-api-access-nbmcj\") pod \"opendatahub-operator-controller-manager-6667d74d96-gc6zv\" (UID: \"a898a04a-8c9c-46f1-b225-f108ebd7fe1b\") " pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:35.596055 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.596002 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a898a04a-8c9c-46f1-b225-f108ebd7fe1b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6667d74d96-gc6zv\" (UID: \"a898a04a-8c9c-46f1-b225-f108ebd7fe1b\") " pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:35.596305 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.596282 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a898a04a-8c9c-46f1-b225-f108ebd7fe1b-manifests\") pod \"opendatahub-operator-controller-manager-6667d74d96-gc6zv\" (UID: \"a898a04a-8c9c-46f1-b225-f108ebd7fe1b\") " pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:35.598324 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.598300 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a898a04a-8c9c-46f1-b225-f108ebd7fe1b-tmp\") pod \"opendatahub-operator-controller-manager-6667d74d96-gc6zv\" (UID: \"a898a04a-8c9c-46f1-b225-f108ebd7fe1b\") " pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:35.598444 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.598383 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a898a04a-8c9c-46f1-b225-f108ebd7fe1b-webhook-cert\") pod \"opendatahub-operator-controller-manager-6667d74d96-gc6zv\" (UID: \"a898a04a-8c9c-46f1-b225-f108ebd7fe1b\") " pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:35.598444 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.598423 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a898a04a-8c9c-46f1-b225-f108ebd7fe1b-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6667d74d96-gc6zv\" (UID: \"a898a04a-8c9c-46f1-b225-f108ebd7fe1b\") " pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:35.603797 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.603775 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbmcj\" (UniqueName: \"kubernetes.io/projected/a898a04a-8c9c-46f1-b225-f108ebd7fe1b-kube-api-access-nbmcj\") pod \"opendatahub-operator-controller-manager-6667d74d96-gc6zv\" (UID: \"a898a04a-8c9c-46f1-b225-f108ebd7fe1b\") " pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:35.759540 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.759450 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:35.896968 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:35.896940 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv"] Jul 18 01:52:35.898722 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:52:35.898693 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda898a04a_8c9c_46f1_b225_f108ebd7fe1b.slice/crio-481e1d4f44be2204f142034d0ca3fd19501abec285a2869a89bb78f65b63fa2e WatchSource:0}: Error finding container 481e1d4f44be2204f142034d0ca3fd19501abec285a2869a89bb78f65b63fa2e: Status 404 returned error can't find the container with id 481e1d4f44be2204f142034d0ca3fd19501abec285a2869a89bb78f65b63fa2e Jul 18 01:52:36.277695 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:36.277633 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" event={"ID":"a898a04a-8c9c-46f1-b225-f108ebd7fe1b","Type":"ContainerStarted","Data":"481e1d4f44be2204f142034d0ca3fd19501abec285a2869a89bb78f65b63fa2e"} Jul 18 01:52:39.292607 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:39.292519 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" event={"ID":"a898a04a-8c9c-46f1-b225-f108ebd7fe1b","Type":"ContainerStarted","Data":"ac91cb5d2af6b2c58100776bca9d7b7ef3b66c19ce3f61e9e25b761b954afc84"} Jul 18 01:52:40.298398 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:40.298364 2570 generic.go:356] "Generic (PLEG): container finished" podID="a898a04a-8c9c-46f1-b225-f108ebd7fe1b" containerID="ac91cb5d2af6b2c58100776bca9d7b7ef3b66c19ce3f61e9e25b761b954afc84" exitCode=0 Jul 18 01:52:40.298821 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:40.298453 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" event={"ID":"a898a04a-8c9c-46f1-b225-f108ebd7fe1b","Type":"ContainerDied","Data":"ac91cb5d2af6b2c58100776bca9d7b7ef3b66c19ce3f61e9e25b761b954afc84"} Jul 18 01:52:40.783336 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:40.783308 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 01:52:40.789786 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:40.789756 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 01:52:40.793919 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:40.793894 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 01:52:40.794066 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:40.794029 2570 kubelet.go:1648] "Image garbage collection succeeded" Jul 18 01:52:40.808500 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:40.808477 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 01:52:41.304102 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:41.304064 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" event={"ID":"a898a04a-8c9c-46f1-b225-f108ebd7fe1b","Type":"ContainerStarted","Data":"28994ef9f64ab975622d1104746226039352f2d2727a606fa1db36977ad545d5"} Jul 18 01:52:41.306463 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:41.304202 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:41.325229 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:41.325156 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" podStartSLOduration=1.893237947 podStartE2EDuration="6.325140923s" totalImagesPullingTime="3.348078868s" totalInitContainerRuntime="1.083824108s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:52:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:52:41.323879395 +0000 UTC m=+301.001303507" watchObservedRunningTime="2026-07-18 01:52:41.325140923 +0000 UTC m=+301.002565035" Jul 18 01:52:52.310834 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:52.310799 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6667d74d96-gc6zv" Jul 18 01:52:55.251698 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.251656 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n"] Jul 18 01:52:55.264401 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.264371 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n" Jul 18 01:52:55.270960 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.270934 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 18 01:52:55.271189 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.271173 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 18 01:52:55.271632 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.271612 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 18 01:52:55.271804 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.271784 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 18 01:52:55.277070 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.277048 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n"] Jul 18 01:52:55.365287 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.365255 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d13fee3-dded-4137-b742-601a6ecb93ae-tmp\") pod \"kube-auth-proxy-7c44dbb944-trg6n\" (UID: \"9d13fee3-dded-4137-b742-601a6ecb93ae\") " pod="openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n" Jul 18 01:52:55.365287 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.365290 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d13fee3-dded-4137-b742-601a6ecb93ae-tls-certs\") pod \"kube-auth-proxy-7c44dbb944-trg6n\" (UID: \"9d13fee3-dded-4137-b742-601a6ecb93ae\") " pod="openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n" Jul 18 01:52:55.365510 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.365343 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcp2r\" (UniqueName: \"kubernetes.io/projected/9d13fee3-dded-4137-b742-601a6ecb93ae-kube-api-access-rcp2r\") pod \"kube-auth-proxy-7c44dbb944-trg6n\" (UID: \"9d13fee3-dded-4137-b742-601a6ecb93ae\") " pod="openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n" Jul 18 01:52:55.464548 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.464328 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jzws6"] Jul 18 01:52:55.470003 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.469968 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d13fee3-dded-4137-b742-601a6ecb93ae-tmp\") pod \"kube-auth-proxy-7c44dbb944-trg6n\" (UID: \"9d13fee3-dded-4137-b742-601a6ecb93ae\") " pod="openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n" Jul 18 01:52:55.470143 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.470013 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d13fee3-dded-4137-b742-601a6ecb93ae-tls-certs\") pod \"kube-auth-proxy-7c44dbb944-trg6n\" (UID: \"9d13fee3-dded-4137-b742-601a6ecb93ae\") " pod="openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n" Jul 18 01:52:55.470143 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.470104 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcp2r\" (UniqueName: \"kubernetes.io/projected/9d13fee3-dded-4137-b742-601a6ecb93ae-kube-api-access-rcp2r\") pod \"kube-auth-proxy-7c44dbb944-trg6n\" (UID: \"9d13fee3-dded-4137-b742-601a6ecb93ae\") " pod="openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n" Jul 18 01:52:55.470232 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:52:55.470190 2570 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 18 01:52:55.470271 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:52:55.470264 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d13fee3-dded-4137-b742-601a6ecb93ae-tls-certs podName:9d13fee3-dded-4137-b742-601a6ecb93ae nodeName:}" failed. No retries permitted until 2026-07-18 01:52:55.970244266 +0000 UTC m=+315.647668356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/9d13fee3-dded-4137-b742-601a6ecb93ae-tls-certs") pod "kube-auth-proxy-7c44dbb944-trg6n" (UID: "9d13fee3-dded-4137-b742-601a6ecb93ae") : secret "kube-auth-proxy-tls" not found Jul 18 01:52:55.470542 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.470517 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jzws6" Jul 18 01:52:55.472496 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.472470 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d13fee3-dded-4137-b742-601a6ecb93ae-tmp\") pod \"kube-auth-proxy-7c44dbb944-trg6n\" (UID: \"9d13fee3-dded-4137-b742-601a6ecb93ae\") " pod="openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n" Jul 18 01:52:55.480054 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.480030 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-7gxks\"" Jul 18 01:52:55.480279 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.480265 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 18 01:52:55.492689 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.492660 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcp2r\" (UniqueName: \"kubernetes.io/projected/9d13fee3-dded-4137-b742-601a6ecb93ae-kube-api-access-rcp2r\") pod \"kube-auth-proxy-7c44dbb944-trg6n\" (UID: \"9d13fee3-dded-4137-b742-601a6ecb93ae\") " pod="openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n" Jul 18 01:52:55.497606 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.497581 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jzws6"] Jul 18 01:52:55.571222 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.571190 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7r2p\" (UniqueName: \"kubernetes.io/projected/e8f8faab-8b16-4476-a4e4-f9056bc8de4b-kube-api-access-f7r2p\") pod \"odh-model-controller-7998c67dc4-jzws6\" (UID: \"e8f8faab-8b16-4476-a4e4-f9056bc8de4b\") " pod="opendatahub/odh-model-controller-7998c67dc4-jzws6" Jul 18 01:52:55.571376 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.571238 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8f8faab-8b16-4476-a4e4-f9056bc8de4b-cert\") pod \"odh-model-controller-7998c67dc4-jzws6\" (UID: \"e8f8faab-8b16-4476-a4e4-f9056bc8de4b\") " pod="opendatahub/odh-model-controller-7998c67dc4-jzws6" Jul 18 01:52:55.671791 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.671756 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7r2p\" (UniqueName: \"kubernetes.io/projected/e8f8faab-8b16-4476-a4e4-f9056bc8de4b-kube-api-access-f7r2p\") pod \"odh-model-controller-7998c67dc4-jzws6\" (UID: \"e8f8faab-8b16-4476-a4e4-f9056bc8de4b\") " pod="opendatahub/odh-model-controller-7998c67dc4-jzws6" Jul 18 01:52:55.672002 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.671817 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8f8faab-8b16-4476-a4e4-f9056bc8de4b-cert\") pod \"odh-model-controller-7998c67dc4-jzws6\" (UID: \"e8f8faab-8b16-4476-a4e4-f9056bc8de4b\") " pod="opendatahub/odh-model-controller-7998c67dc4-jzws6" Jul 18 01:52:55.672002 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:52:55.671944 2570 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 18 01:52:55.672002 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:52:55.671996 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8f8faab-8b16-4476-a4e4-f9056bc8de4b-cert podName:e8f8faab-8b16-4476-a4e4-f9056bc8de4b nodeName:}" failed. No retries permitted until 2026-07-18 01:52:56.1719832 +0000 UTC m=+315.849407289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/e8f8faab-8b16-4476-a4e4-f9056bc8de4b-cert") pod "odh-model-controller-7998c67dc4-jzws6" (UID: "e8f8faab-8b16-4476-a4e4-f9056bc8de4b") : secret "odh-model-controller-webhook-cert" not found Jul 18 01:52:55.714320 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.714263 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7r2p\" (UniqueName: \"kubernetes.io/projected/e8f8faab-8b16-4476-a4e4-f9056bc8de4b-kube-api-access-f7r2p\") pod \"odh-model-controller-7998c67dc4-jzws6\" (UID: \"e8f8faab-8b16-4476-a4e4-f9056bc8de4b\") " pod="opendatahub/odh-model-controller-7998c67dc4-jzws6" Jul 18 01:52:55.973598 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.973504 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d13fee3-dded-4137-b742-601a6ecb93ae-tls-certs\") pod \"kube-auth-proxy-7c44dbb944-trg6n\" (UID: \"9d13fee3-dded-4137-b742-601a6ecb93ae\") " pod="openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n" Jul 18 01:52:55.975990 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:55.975956 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d13fee3-dded-4137-b742-601a6ecb93ae-tls-certs\") pod \"kube-auth-proxy-7c44dbb944-trg6n\" (UID: \"9d13fee3-dded-4137-b742-601a6ecb93ae\") " pod="openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n" Jul 18 01:52:56.174385 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.174346 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n" Jul 18 01:52:56.174589 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.174568 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8f8faab-8b16-4476-a4e4-f9056bc8de4b-cert\") pod \"odh-model-controller-7998c67dc4-jzws6\" (UID: \"e8f8faab-8b16-4476-a4e4-f9056bc8de4b\") " pod="opendatahub/odh-model-controller-7998c67dc4-jzws6" Jul 18 01:52:56.177054 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.177029 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8f8faab-8b16-4476-a4e4-f9056bc8de4b-cert\") pod \"odh-model-controller-7998c67dc4-jzws6\" (UID: \"e8f8faab-8b16-4476-a4e4-f9056bc8de4b\") " pod="opendatahub/odh-model-controller-7998c67dc4-jzws6" Jul 18 01:52:56.305906 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.305877 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n"] Jul 18 01:52:56.308985 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:52:56.308961 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d13fee3_dded_4137_b742_601a6ecb93ae.slice/crio-acc02e144de3afd66daa35256dc36a5258234bf7ff7b810f2f4193bb6755786b WatchSource:0}: Error finding container acc02e144de3afd66daa35256dc36a5258234bf7ff7b810f2f4193bb6755786b: Status 404 returned error can't find the container with id acc02e144de3afd66daa35256dc36a5258234bf7ff7b810f2f4193bb6755786b Jul 18 01:52:56.310710 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.310695 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 01:52:56.366461 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.366427 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n" event={"ID":"9d13fee3-dded-4137-b742-601a6ecb93ae","Type":"ContainerStarted","Data":"acc02e144de3afd66daa35256dc36a5258234bf7ff7b810f2f4193bb6755786b"} Jul 18 01:52:56.386865 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.386833 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-jzws6" Jul 18 01:52:56.522911 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.522838 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-jzws6"] Jul 18 01:52:56.525794 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:52:56.525768 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8f8faab_8b16_4476_a4e4_f9056bc8de4b.slice/crio-0f93e5ce2bb153441412d7f73e7bb2e79be5ad80088b8e0ff7d229db7629771f WatchSource:0}: Error finding container 0f93e5ce2bb153441412d7f73e7bb2e79be5ad80088b8e0ff7d229db7629771f: Status 404 returned error can't find the container with id 0f93e5ce2bb153441412d7f73e7bb2e79be5ad80088b8e0ff7d229db7629771f Jul 18 01:52:56.636470 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.636430 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5w78s"] Jul 18 01:52:56.641330 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.641305 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5w78s" Jul 18 01:52:56.645340 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.645309 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 18 01:52:56.660737 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.660712 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5w78s"] Jul 18 01:52:56.780096 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.780006 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4298078f-366e-4871-be68-fda580328746-tls-certs\") pod \"model-serving-api-54d89bd5fc-5w78s\" (UID: \"4298078f-366e-4871-be68-fda580328746\") " pod="opendatahub/model-serving-api-54d89bd5fc-5w78s" Jul 18 01:52:56.780096 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.780072 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbnkt\" (UniqueName: \"kubernetes.io/projected/4298078f-366e-4871-be68-fda580328746-kube-api-access-rbnkt\") pod \"model-serving-api-54d89bd5fc-5w78s\" (UID: \"4298078f-366e-4871-be68-fda580328746\") " pod="opendatahub/model-serving-api-54d89bd5fc-5w78s" Jul 18 01:52:56.880907 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.880869 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4298078f-366e-4871-be68-fda580328746-tls-certs\") pod \"model-serving-api-54d89bd5fc-5w78s\" (UID: \"4298078f-366e-4871-be68-fda580328746\") " pod="opendatahub/model-serving-api-54d89bd5fc-5w78s" Jul 18 01:52:56.881098 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.880933 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbnkt\" (UniqueName: \"kubernetes.io/projected/4298078f-366e-4871-be68-fda580328746-kube-api-access-rbnkt\") pod \"model-serving-api-54d89bd5fc-5w78s\" (UID: \"4298078f-366e-4871-be68-fda580328746\") " pod="opendatahub/model-serving-api-54d89bd5fc-5w78s" Jul 18 01:52:56.884188 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.884149 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4298078f-366e-4871-be68-fda580328746-tls-certs\") pod \"model-serving-api-54d89bd5fc-5w78s\" (UID: \"4298078f-366e-4871-be68-fda580328746\") " pod="opendatahub/model-serving-api-54d89bd5fc-5w78s" Jul 18 01:52:56.899354 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.899315 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbnkt\" (UniqueName: \"kubernetes.io/projected/4298078f-366e-4871-be68-fda580328746-kube-api-access-rbnkt\") pod \"model-serving-api-54d89bd5fc-5w78s\" (UID: \"4298078f-366e-4871-be68-fda580328746\") " pod="opendatahub/model-serving-api-54d89bd5fc-5w78s" Jul 18 01:52:56.951207 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:56.951176 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5w78s" Jul 18 01:52:57.115332 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.115304 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5w78s"] Jul 18 01:52:57.117257 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:52:57.117229 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4298078f_366e_4871_be68_fda580328746.slice/crio-1da4b686c2bb867d5e8aa8471d38001aaad053030fcc64143acd451eb4cf63cf WatchSource:0}: Error finding container 1da4b686c2bb867d5e8aa8471d38001aaad053030fcc64143acd451eb4cf63cf: Status 404 returned error can't find the container with id 1da4b686c2bb867d5e8aa8471d38001aaad053030fcc64143acd451eb4cf63cf Jul 18 01:52:57.238994 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.238955 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr"] Jul 18 01:52:57.243691 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.243670 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.246654 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.246623 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 18 01:52:57.246866 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.246845 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-kph2k\"" Jul 18 01:52:57.246953 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.246880 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 18 01:52:57.250476 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.250452 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 18 01:52:57.250585 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.250527 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 18 01:52:57.276016 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.275987 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr"] Jul 18 01:52:57.372008 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.371976 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5w78s" event={"ID":"4298078f-366e-4871-be68-fda580328746","Type":"ContainerStarted","Data":"1da4b686c2bb867d5e8aa8471d38001aaad053030fcc64143acd451eb4cf63cf"} Jul 18 01:52:57.373115 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.373086 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jzws6" event={"ID":"e8f8faab-8b16-4476-a4e4-f9056bc8de4b","Type":"ContainerStarted","Data":"0f93e5ce2bb153441412d7f73e7bb2e79be5ad80088b8e0ff7d229db7629771f"} Jul 18 01:52:57.384551 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.384523 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.384676 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.384568 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkb26\" (UniqueName: \"kubernetes.io/projected/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-kube-api-access-nkb26\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.384676 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.384605 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-cacerts\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.384790 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.384681 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.384790 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.384700 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-local-certs\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.384790 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.384726 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.384911 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.384795 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-token\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.485313 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.485282 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.485494 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.485319 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkb26\" (UniqueName: \"kubernetes.io/projected/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-kube-api-access-nkb26\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.485494 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.485350 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-cacerts\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.485494 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.485380 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.485494 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.485404 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-local-certs\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.485494 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.485435 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.485784 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.485561 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-token\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.486241 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.486218 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.487853 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.487826 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-cacerts\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.487950 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.487931 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.488021 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.488004 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-local-certs\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.488056 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.488024 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.506459 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.506440 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkb26\" (UniqueName: \"kubernetes.io/projected/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-kube-api-access-nkb26\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.509726 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.509707 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-token\") pod \"istiod-openshift-gateway-7574459656-vv2nr\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.555475 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.555443 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:52:57.738485 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:57.738384 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr"] Jul 18 01:52:57.741276 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:52:57.741246 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fe0b7e9_deea_4069_bb4a_cb1a31764dbd.slice/crio-2b2208d68613b060de0c89dbcf0c98a64958f0336f9d1c27bd81bd6ec06639a3 WatchSource:0}: Error finding container 2b2208d68613b060de0c89dbcf0c98a64958f0336f9d1c27bd81bd6ec06639a3: Status 404 returned error can't find the container with id 2b2208d68613b060de0c89dbcf0c98a64958f0336f9d1c27bd81bd6ec06639a3 Jul 18 01:52:58.381298 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:52:58.381207 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" event={"ID":"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd","Type":"ContainerStarted","Data":"2b2208d68613b060de0c89dbcf0c98a64958f0336f9d1c27bd81bd6ec06639a3"} Jul 18 01:53:01.396021 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:01.395977 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n" event={"ID":"9d13fee3-dded-4137-b742-601a6ecb93ae","Type":"ContainerStarted","Data":"6163ff656415d4b9c88cd6f03896294652dfa0487c95e6a4b4cfa6e69bb52887"} Jul 18 01:53:01.397700 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:01.397665 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5w78s" event={"ID":"4298078f-366e-4871-be68-fda580328746","Type":"ContainerStarted","Data":"717595c20e2269c9c7a1dca5379c8dd9386d383021d5a10faa159d7b982a8302"} Jul 18 01:53:01.397852 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:01.397744 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-5w78s" Jul 18 01:53:01.399269 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:01.399243 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-jzws6" event={"ID":"e8f8faab-8b16-4476-a4e4-f9056bc8de4b","Type":"ContainerStarted","Data":"49042f33578dacfa470eae49bcc98319351a1dbee9d3909a859f4891c534b3f6"} Jul 18 01:53:01.399415 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:01.399389 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-jzws6" Jul 18 01:53:01.416123 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:01.416065 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c44dbb944-trg6n" podStartSLOduration=2.141534539 podStartE2EDuration="6.41605228s" totalImagesPullingTime="4.274517741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:01.414669338 +0000 UTC m=+321.092093452" watchObservedRunningTime="2026-07-18 01:53:01.41605228 +0000 UTC m=+321.093476391" Jul 18 01:53:01.433082 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:01.433025 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-5w78s" podStartSLOduration=1.967195743 podStartE2EDuration="5.433005962s" totalImagesPullingTime="3.465810219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:52:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:01.432234883 +0000 UTC m=+321.109658994" watchObservedRunningTime="2026-07-18 01:53:01.433005962 +0000 UTC m=+321.110430074" Jul 18 01:53:01.457969 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:01.457907 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-jzws6" podStartSLOduration=2.350380402 podStartE2EDuration="6.457886063s" totalImagesPullingTime="4.107505661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:52:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:01.456861334 +0000 UTC m=+321.134285449" watchObservedRunningTime="2026-07-18 01:53:01.457886063 +0000 UTC m=+321.135310176" Jul 18 01:53:01.958044 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:01.958006 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:53:01.958120 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:01.958083 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:53:02.404417 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:02.404379 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" event={"ID":"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd","Type":"ContainerStarted","Data":"4d4c9861b2d9cac54efc9a49b8d19ea014c5a64500cdd769cbdd4066cc1e5f06"} Jul 18 01:53:02.404931 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:02.404911 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:53:02.426565 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:02.426508 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" podStartSLOduration=1.213489898 podStartE2EDuration="5.426487849s" totalImagesPullingTime="4.212997951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:52:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:02.425835744 +0000 UTC m=+322.103259854" watchObservedRunningTime="2026-07-18 01:53:02.426487849 +0000 UTC m=+322.103911960" Jul 18 01:53:03.410522 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:03.410484 2570 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-vv2nr container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 18 01:53:03.410902 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:03.410543 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" podUID="0fe0b7e9-deea-4069-bb4a-cb1a31764dbd" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 18 01:53:03.803521 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:03.803488 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj"] Jul 18 01:53:03.806803 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:03.806786 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:03.810057 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:03.810020 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-9brtk\"" Jul 18 01:53:03.820136 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:03.820111 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj"] Jul 18 01:53:03.947187 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:03.947145 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7ppx\" (UniqueName: \"kubernetes.io/projected/53a4628d-9ba5-4d60-8a90-3046f2b24220-kube-api-access-s7ppx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:03.947370 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:03.947243 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53a4628d-9ba5-4d60-8a90-3046f2b24220-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:03.947370 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:03.947314 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53a4628d-9ba5-4d60-8a90-3046f2b24220-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:03.947488 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:03.947468 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53a4628d-9ba5-4d60-8a90-3046f2b24220-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:03.947543 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:03.947505 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53a4628d-9ba5-4d60-8a90-3046f2b24220-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:03.947614 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:03.947595 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53a4628d-9ba5-4d60-8a90-3046f2b24220-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:03.947688 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:03.947662 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53a4628d-9ba5-4d60-8a90-3046f2b24220-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:03.947727 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:03.947694 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53a4628d-9ba5-4d60-8a90-3046f2b24220-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:03.947769 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:03.947739 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53a4628d-9ba5-4d60-8a90-3046f2b24220-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.048782 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.048734 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53a4628d-9ba5-4d60-8a90-3046f2b24220-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.048971 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.048888 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53a4628d-9ba5-4d60-8a90-3046f2b24220-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.048971 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.048931 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53a4628d-9ba5-4d60-8a90-3046f2b24220-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.048971 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.048950 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53a4628d-9ba5-4d60-8a90-3046f2b24220-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.049125 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.048994 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53a4628d-9ba5-4d60-8a90-3046f2b24220-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.049125 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.049014 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53a4628d-9ba5-4d60-8a90-3046f2b24220-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.049125 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.049033 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53a4628d-9ba5-4d60-8a90-3046f2b24220-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.049125 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.049078 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53a4628d-9ba5-4d60-8a90-3046f2b24220-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.049319 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.049139 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7ppx\" (UniqueName: \"kubernetes.io/projected/53a4628d-9ba5-4d60-8a90-3046f2b24220-kube-api-access-s7ppx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.049319 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.049280 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/53a4628d-9ba5-4d60-8a90-3046f2b24220-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.049764 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.049739 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/53a4628d-9ba5-4d60-8a90-3046f2b24220-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.049926 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.049826 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/53a4628d-9ba5-4d60-8a90-3046f2b24220-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.049926 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.049920 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/53a4628d-9ba5-4d60-8a90-3046f2b24220-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.049926 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.049924 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/53a4628d-9ba5-4d60-8a90-3046f2b24220-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.051267 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.051242 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/53a4628d-9ba5-4d60-8a90-3046f2b24220-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.051732 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.051714 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/53a4628d-9ba5-4d60-8a90-3046f2b24220-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.058624 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.058564 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/53a4628d-9ba5-4d60-8a90-3046f2b24220-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.059897 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.059877 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7ppx\" (UniqueName: \"kubernetes.io/projected/53a4628d-9ba5-4d60-8a90-3046f2b24220-kube-api-access-s7ppx\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xh8tj\" (UID: \"53a4628d-9ba5-4d60-8a90-3046f2b24220\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.118439 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.118403 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:04.292872 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.292847 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj"] Jul 18 01:53:04.294820 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:53:04.294799 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53a4628d_9ba5_4d60_8a90_3046f2b24220.slice/crio-c352f3ff2925ae4ead822dc374f2b6b5fd84e0171ab385df9f95f6563e4cbaf3 WatchSource:0}: Error finding container c352f3ff2925ae4ead822dc374f2b6b5fd84e0171ab385df9f95f6563e4cbaf3: Status 404 returned error can't find the container with id c352f3ff2925ae4ead822dc374f2b6b5fd84e0171ab385df9f95f6563e4cbaf3 Jul 18 01:53:04.413764 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.413683 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" event={"ID":"53a4628d-9ba5-4d60-8a90-3046f2b24220","Type":"ContainerStarted","Data":"c352f3ff2925ae4ead822dc374f2b6b5fd84e0171ab385df9f95f6563e4cbaf3"} Jul 18 01:53:04.414417 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:04.414399 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:53:05.492393 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.492357 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gw8pr"] Jul 18 01:53:05.496168 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.496122 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gw8pr" Jul 18 01:53:05.498608 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.498583 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 18 01:53:05.498992 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.498973 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-bxcnq\"" Jul 18 01:53:05.508853 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.508754 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ffh2m"] Jul 18 01:53:05.512213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.512196 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffh2m" Jul 18 01:53:05.516691 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.516618 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gw8pr"] Jul 18 01:53:05.518020 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.517998 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-g9tk4\"" Jul 18 01:53:05.518843 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.518694 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 18 01:53:05.528262 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.528242 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ffh2m"] Jul 18 01:53:05.563829 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.563802 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b695q\" (UniqueName: \"kubernetes.io/projected/520b5ebf-6700-4b39-97c6-1c364f0b9987-kube-api-access-b695q\") pod \"llmisvc-controller-manager-777c6bd7d5-ffh2m\" (UID: \"520b5ebf-6700-4b39-97c6-1c364f0b9987\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffh2m" Jul 18 01:53:05.563979 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.563854 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2zsz\" (UniqueName: \"kubernetes.io/projected/51a99fed-12aa-4a7d-a4d0-238bbb5454b1-kube-api-access-b2zsz\") pod \"kserve-controller-manager-d6f4b65b8-gw8pr\" (UID: \"51a99fed-12aa-4a7d-a4d0-238bbb5454b1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gw8pr" Jul 18 01:53:05.563979 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.563905 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/520b5ebf-6700-4b39-97c6-1c364f0b9987-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ffh2m\" (UID: \"520b5ebf-6700-4b39-97c6-1c364f0b9987\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffh2m" Jul 18 01:53:05.564094 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.563988 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51a99fed-12aa-4a7d-a4d0-238bbb5454b1-cert\") pod \"kserve-controller-manager-d6f4b65b8-gw8pr\" (UID: \"51a99fed-12aa-4a7d-a4d0-238bbb5454b1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gw8pr" Jul 18 01:53:05.665252 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.665214 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51a99fed-12aa-4a7d-a4d0-238bbb5454b1-cert\") pod \"kserve-controller-manager-d6f4b65b8-gw8pr\" (UID: \"51a99fed-12aa-4a7d-a4d0-238bbb5454b1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gw8pr" Jul 18 01:53:05.665451 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.665278 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b695q\" (UniqueName: \"kubernetes.io/projected/520b5ebf-6700-4b39-97c6-1c364f0b9987-kube-api-access-b695q\") pod \"llmisvc-controller-manager-777c6bd7d5-ffh2m\" (UID: \"520b5ebf-6700-4b39-97c6-1c364f0b9987\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffh2m" Jul 18 01:53:05.665451 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:53:05.665394 2570 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 18 01:53:05.665697 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:53:05.665463 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51a99fed-12aa-4a7d-a4d0-238bbb5454b1-cert podName:51a99fed-12aa-4a7d-a4d0-238bbb5454b1 nodeName:}" failed. No retries permitted until 2026-07-18 01:53:06.1654414 +0000 UTC m=+325.842865504 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/51a99fed-12aa-4a7d-a4d0-238bbb5454b1-cert") pod "kserve-controller-manager-d6f4b65b8-gw8pr" (UID: "51a99fed-12aa-4a7d-a4d0-238bbb5454b1") : secret "kserve-webhook-server-cert" not found Jul 18 01:53:05.665697 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.665555 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2zsz\" (UniqueName: \"kubernetes.io/projected/51a99fed-12aa-4a7d-a4d0-238bbb5454b1-kube-api-access-b2zsz\") pod \"kserve-controller-manager-d6f4b65b8-gw8pr\" (UID: \"51a99fed-12aa-4a7d-a4d0-238bbb5454b1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gw8pr" Jul 18 01:53:05.665697 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.665597 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/520b5ebf-6700-4b39-97c6-1c364f0b9987-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ffh2m\" (UID: \"520b5ebf-6700-4b39-97c6-1c364f0b9987\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffh2m" Jul 18 01:53:05.665897 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:53:05.665735 2570 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 18 01:53:05.665897 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:53:05.665777 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/520b5ebf-6700-4b39-97c6-1c364f0b9987-cert podName:520b5ebf-6700-4b39-97c6-1c364f0b9987 nodeName:}" failed. No retries permitted until 2026-07-18 01:53:06.165761545 +0000 UTC m=+325.843185635 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/520b5ebf-6700-4b39-97c6-1c364f0b9987-cert") pod "llmisvc-controller-manager-777c6bd7d5-ffh2m" (UID: "520b5ebf-6700-4b39-97c6-1c364f0b9987") : secret "llmisvc-webhook-server-cert" not found Jul 18 01:53:05.684705 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.684628 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b695q\" (UniqueName: \"kubernetes.io/projected/520b5ebf-6700-4b39-97c6-1c364f0b9987-kube-api-access-b695q\") pod \"llmisvc-controller-manager-777c6bd7d5-ffh2m\" (UID: \"520b5ebf-6700-4b39-97c6-1c364f0b9987\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffh2m" Jul 18 01:53:05.684871 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:05.684751 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2zsz\" (UniqueName: \"kubernetes.io/projected/51a99fed-12aa-4a7d-a4d0-238bbb5454b1-kube-api-access-b2zsz\") pod \"kserve-controller-manager-d6f4b65b8-gw8pr\" (UID: \"51a99fed-12aa-4a7d-a4d0-238bbb5454b1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gw8pr" Jul 18 01:53:06.171086 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:06.171050 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/520b5ebf-6700-4b39-97c6-1c364f0b9987-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ffh2m\" (UID: \"520b5ebf-6700-4b39-97c6-1c364f0b9987\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffh2m" Jul 18 01:53:06.171273 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:06.171222 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51a99fed-12aa-4a7d-a4d0-238bbb5454b1-cert\") pod \"kserve-controller-manager-d6f4b65b8-gw8pr\" (UID: \"51a99fed-12aa-4a7d-a4d0-238bbb5454b1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gw8pr" Jul 18 01:53:06.173722 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:06.173695 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51a99fed-12aa-4a7d-a4d0-238bbb5454b1-cert\") pod \"kserve-controller-manager-d6f4b65b8-gw8pr\" (UID: \"51a99fed-12aa-4a7d-a4d0-238bbb5454b1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gw8pr" Jul 18 01:53:06.173839 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:06.173755 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/520b5ebf-6700-4b39-97c6-1c364f0b9987-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ffh2m\" (UID: \"520b5ebf-6700-4b39-97c6-1c364f0b9987\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffh2m" Jul 18 01:53:06.411479 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:06.411445 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gw8pr" Jul 18 01:53:06.429136 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:06.429070 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffh2m" Jul 18 01:53:07.188890 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.188855 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr"] Jul 18 01:53:07.189335 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.189087 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" podUID="0fe0b7e9-deea-4069-bb4a-cb1a31764dbd" containerName="discovery" containerID="cri-o://4d4c9861b2d9cac54efc9a49b8d19ea014c5a64500cdd769cbdd4066cc1e5f06" gracePeriod=30 Jul 18 01:53:07.295934 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.295126 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:53:07.295934 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.295227 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:53:07.295934 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.295274 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:53:07.380211 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.379126 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ffh2m"] Jul 18 01:53:07.382805 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:53:07.382706 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod520b5ebf_6700_4b39_97c6_1c364f0b9987.slice/crio-bfecd6b130b973571803b75772a78c33a3b81f19802256f08393f59cb4907c60 WatchSource:0}: Error finding container bfecd6b130b973571803b75772a78c33a3b81f19802256f08393f59cb4907c60: Status 404 returned error can't find the container with id bfecd6b130b973571803b75772a78c33a3b81f19802256f08393f59cb4907c60 Jul 18 01:53:07.410027 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.409948 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gw8pr"] Jul 18 01:53:07.410937 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:53:07.410909 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51a99fed_12aa_4a7d_a4d0_238bbb5454b1.slice/crio-b376a440ee264d49f834d59e8c47bd19e49bdc0755adbbba83777c2ee45a5c60 WatchSource:0}: Error finding container b376a440ee264d49f834d59e8c47bd19e49bdc0755adbbba83777c2ee45a5c60: Status 404 returned error can't find the container with id b376a440ee264d49f834d59e8c47bd19e49bdc0755adbbba83777c2ee45a5c60 Jul 18 01:53:07.431261 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.431229 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffh2m" event={"ID":"520b5ebf-6700-4b39-97c6-1c364f0b9987","Type":"ContainerStarted","Data":"bfecd6b130b973571803b75772a78c33a3b81f19802256f08393f59cb4907c60"} Jul 18 01:53:07.433211 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.433172 2570 generic.go:356] "Generic (PLEG): container finished" podID="0fe0b7e9-deea-4069-bb4a-cb1a31764dbd" containerID="4d4c9861b2d9cac54efc9a49b8d19ea014c5a64500cdd769cbdd4066cc1e5f06" exitCode=0 Jul 18 01:53:07.433342 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.433240 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" event={"ID":"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd","Type":"ContainerDied","Data":"4d4c9861b2d9cac54efc9a49b8d19ea014c5a64500cdd769cbdd4066cc1e5f06"} Jul 18 01:53:07.434878 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.434851 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" event={"ID":"53a4628d-9ba5-4d60-8a90-3046f2b24220","Type":"ContainerStarted","Data":"092df97c86af629ae43e08d38e0d6dd3f4b7ed677b413f6d52e4400c84b234fd"} Jul 18 01:53:07.436444 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.436379 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gw8pr" event={"ID":"51a99fed-12aa-4a7d-a4d0-238bbb5454b1","Type":"ContainerStarted","Data":"b376a440ee264d49f834d59e8c47bd19e49bdc0755adbbba83777c2ee45a5c60"} Jul 18 01:53:07.472313 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.472289 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:53:07.488365 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.488302 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" podStartSLOduration=1.490535106 podStartE2EDuration="4.488287886s" totalImagesPullingTime="2.99775278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:07.486740891 +0000 UTC m=+327.164165005" watchObservedRunningTime="2026-07-18 01:53:07.488287886 +0000 UTC m=+327.165711997" Jul 18 01:53:07.586164 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.586131 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-cacerts\") pod \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " Jul 18 01:53:07.586328 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.586175 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-csr-ca-configmap\") pod \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " Jul 18 01:53:07.586328 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.586246 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-csr-dns-cert\") pod \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " Jul 18 01:53:07.586328 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.586298 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkb26\" (UniqueName: \"kubernetes.io/projected/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-kube-api-access-nkb26\") pod \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " Jul 18 01:53:07.586473 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.586330 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-token\") pod \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " Jul 18 01:53:07.586473 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.586355 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-kubeconfig\") pod \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " Jul 18 01:53:07.586473 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.586380 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-local-certs\") pod \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\" (UID: \"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd\") " Jul 18 01:53:07.586654 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.586577 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "0fe0b7e9-deea-4069-bb4a-cb1a31764dbd" (UID: "0fe0b7e9-deea-4069-bb4a-cb1a31764dbd"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:53:07.588959 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.588927 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-local-certs" (OuterVolumeSpecName: "local-certs") pod "0fe0b7e9-deea-4069-bb4a-cb1a31764dbd" (UID: "0fe0b7e9-deea-4069-bb4a-cb1a31764dbd"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:53:07.589089 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.588988 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-cacerts" (OuterVolumeSpecName: "cacerts") pod "0fe0b7e9-deea-4069-bb4a-cb1a31764dbd" (UID: "0fe0b7e9-deea-4069-bb4a-cb1a31764dbd"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:53:07.589089 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.589073 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "0fe0b7e9-deea-4069-bb4a-cb1a31764dbd" (UID: "0fe0b7e9-deea-4069-bb4a-cb1a31764dbd"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:53:07.589089 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.589080 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-kube-api-access-nkb26" (OuterVolumeSpecName: "kube-api-access-nkb26") pod "0fe0b7e9-deea-4069-bb4a-cb1a31764dbd" (UID: "0fe0b7e9-deea-4069-bb4a-cb1a31764dbd"). InnerVolumeSpecName "kube-api-access-nkb26". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:53:07.589206 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.589088 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "0fe0b7e9-deea-4069-bb4a-cb1a31764dbd" (UID: "0fe0b7e9-deea-4069-bb4a-cb1a31764dbd"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:53:07.589206 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.589090 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-token" (OuterVolumeSpecName: "istio-token") pod "0fe0b7e9-deea-4069-bb4a-cb1a31764dbd" (UID: "0fe0b7e9-deea-4069-bb4a-cb1a31764dbd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:53:07.687296 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.687262 2570 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-csr-dns-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:53:07.687296 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.687289 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkb26\" (UniqueName: \"kubernetes.io/projected/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-kube-api-access-nkb26\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:53:07.687492 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.687304 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-token\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:53:07.687492 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.687316 2570 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-kubeconfig\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:53:07.687492 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.687327 2570 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-local-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:53:07.687492 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.687338 2570 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-cacerts\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:53:07.687492 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:07.687348 2570 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd-istio-csr-ca-configmap\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:53:08.118585 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:08.118549 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:08.122927 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:08.122905 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:08.444632 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:08.444553 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" Jul 18 01:53:08.444632 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:08.444577 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" event={"ID":"0fe0b7e9-deea-4069-bb4a-cb1a31764dbd","Type":"ContainerDied","Data":"2b2208d68613b060de0c89dbcf0c98a64958f0336f9d1c27bd81bd6ec06639a3"} Jul 18 01:53:08.445174 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:08.444658 2570 scope.go:117] "RemoveContainer" containerID="4d4c9861b2d9cac54efc9a49b8d19ea014c5a64500cdd769cbdd4066cc1e5f06" Jul 18 01:53:08.445174 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:08.444959 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:08.446163 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:08.445911 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xh8tj" Jul 18 01:53:08.492707 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:08.492111 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr"] Jul 18 01:53:08.498155 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:08.498121 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr"] Jul 18 01:53:08.894814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:08.894777 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fe0b7e9-deea-4069-bb4a-cb1a31764dbd" path="/var/lib/kubelet/pods/0fe0b7e9-deea-4069-bb4a-cb1a31764dbd/volumes" Jul 18 01:53:11.463525 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:11.463484 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffh2m" event={"ID":"520b5ebf-6700-4b39-97c6-1c364f0b9987","Type":"ContainerStarted","Data":"794e31080defd46549aafcc0214dec8a734707f259c2eadf0e683d90b57c745f"} Jul 18 01:53:11.463997 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:11.463572 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffh2m" Jul 18 01:53:11.465000 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:11.464977 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gw8pr" event={"ID":"51a99fed-12aa-4a7d-a4d0-238bbb5454b1","Type":"ContainerStarted","Data":"a3482ae1f955901ea4b69d68d202edbd31af88144829d213a97694733bc33c10"} Jul 18 01:53:11.465126 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:11.465091 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gw8pr" Jul 18 01:53:11.480881 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:11.480839 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffh2m" podStartSLOduration=3.427747527 podStartE2EDuration="6.480825611s" totalImagesPullingTime="3.053078084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:11.479182248 +0000 UTC m=+331.156606361" watchObservedRunningTime="2026-07-18 01:53:11.480825611 +0000 UTC m=+331.158249722" Jul 18 01:53:11.495154 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:11.495115 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gw8pr" podStartSLOduration=3.5149795299999997 podStartE2EDuration="6.495103009s" totalImagesPullingTime="2.980123479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:11.493500395 +0000 UTC m=+331.170924520" watchObservedRunningTime="2026-07-18 01:53:11.495103009 +0000 UTC m=+331.172527119" Jul 18 01:53:12.406797 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:12.406767 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-jzws6" Jul 18 01:53:12.408511 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:12.408494 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-5w78s" Jul 18 01:53:12.414990 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:12.414969 2570 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-vv2nr container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.27:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 18 01:53:12.415063 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:12.415014 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-vv2nr" podUID="0fe0b7e9-deea-4069-bb4a-cb1a31764dbd" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.27:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 18 01:53:42.472285 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:42.472251 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ffh2m" Jul 18 01:53:42.474552 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:42.474532 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gw8pr" Jul 18 01:53:56.259929 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.259895 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nbcc6"] Jul 18 01:53:56.260513 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.260383 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fe0b7e9-deea-4069-bb4a-cb1a31764dbd" containerName="discovery" Jul 18 01:53:56.260513 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.260403 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fe0b7e9-deea-4069-bb4a-cb1a31764dbd" containerName="discovery" Jul 18 01:53:56.260513 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.260494 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fe0b7e9-deea-4069-bb4a-cb1a31764dbd" containerName="discovery" Jul 18 01:53:56.265163 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.265142 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nbcc6" Jul 18 01:53:56.268132 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.268110 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-hc9tb\"" Jul 18 01:53:56.268855 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.268835 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 18 01:53:56.269017 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.268889 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 18 01:53:56.278828 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.278809 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nbcc6"] Jul 18 01:53:56.308248 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.308217 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtg8v\" (UniqueName: \"kubernetes.io/projected/9273d20a-478a-4472-afd4-218bb890a4b6-kube-api-access-mtg8v\") pod \"kuadrant-operator-catalog-nbcc6\" (UID: \"9273d20a-478a-4472-afd4-218bb890a4b6\") " pod="kuadrant-system/kuadrant-operator-catalog-nbcc6" Jul 18 01:53:56.408693 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.408662 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtg8v\" (UniqueName: \"kubernetes.io/projected/9273d20a-478a-4472-afd4-218bb890a4b6-kube-api-access-mtg8v\") pod \"kuadrant-operator-catalog-nbcc6\" (UID: \"9273d20a-478a-4472-afd4-218bb890a4b6\") " pod="kuadrant-system/kuadrant-operator-catalog-nbcc6" Jul 18 01:53:56.417335 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.417308 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtg8v\" (UniqueName: \"kubernetes.io/projected/9273d20a-478a-4472-afd4-218bb890a4b6-kube-api-access-mtg8v\") pod \"kuadrant-operator-catalog-nbcc6\" (UID: \"9273d20a-478a-4472-afd4-218bb890a4b6\") " pod="kuadrant-system/kuadrant-operator-catalog-nbcc6" Jul 18 01:53:56.575986 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.575950 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nbcc6" Jul 18 01:53:56.628401 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.628364 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nbcc6"] Jul 18 01:53:56.710454 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.710420 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nbcc6"] Jul 18 01:53:56.713000 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:53:56.712971 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9273d20a_478a_4472_afd4_218bb890a4b6.slice/crio-87bdf9aac754605adb4712d268097b70e432254b37f28369d1e4cb2b5f26edb3 WatchSource:0}: Error finding container 87bdf9aac754605adb4712d268097b70e432254b37f28369d1e4cb2b5f26edb3: Status 404 returned error can't find the container with id 87bdf9aac754605adb4712d268097b70e432254b37f28369d1e4cb2b5f26edb3 Jul 18 01:53:56.835710 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.835608 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fgrxl"] Jul 18 01:53:56.840128 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.840112 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fgrxl" Jul 18 01:53:56.846549 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.846522 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fgrxl"] Jul 18 01:53:56.912229 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:56.912201 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl4vs\" (UniqueName: \"kubernetes.io/projected/eb17e69e-d29d-4ec2-9443-2509b6b827e0-kube-api-access-bl4vs\") pod \"kuadrant-operator-catalog-fgrxl\" (UID: \"eb17e69e-d29d-4ec2-9443-2509b6b827e0\") " pod="kuadrant-system/kuadrant-operator-catalog-fgrxl" Jul 18 01:53:57.013421 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:57.013386 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl4vs\" (UniqueName: \"kubernetes.io/projected/eb17e69e-d29d-4ec2-9443-2509b6b827e0-kube-api-access-bl4vs\") pod \"kuadrant-operator-catalog-fgrxl\" (UID: \"eb17e69e-d29d-4ec2-9443-2509b6b827e0\") " pod="kuadrant-system/kuadrant-operator-catalog-fgrxl" Jul 18 01:53:57.021627 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:57.021594 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl4vs\" (UniqueName: \"kubernetes.io/projected/eb17e69e-d29d-4ec2-9443-2509b6b827e0-kube-api-access-bl4vs\") pod \"kuadrant-operator-catalog-fgrxl\" (UID: \"eb17e69e-d29d-4ec2-9443-2509b6b827e0\") " pod="kuadrant-system/kuadrant-operator-catalog-fgrxl" Jul 18 01:53:57.150056 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:57.149965 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-fgrxl" Jul 18 01:53:57.293417 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:57.293383 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-fgrxl"] Jul 18 01:53:57.316097 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:53:57.316056 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb17e69e_d29d_4ec2_9443_2509b6b827e0.slice/crio-0fcf48bf2560002479937783fed7732201371d35065a3bb61e0381c7cdacda5f WatchSource:0}: Error finding container 0fcf48bf2560002479937783fed7732201371d35065a3bb61e0381c7cdacda5f: Status 404 returned error can't find the container with id 0fcf48bf2560002479937783fed7732201371d35065a3bb61e0381c7cdacda5f Jul 18 01:53:57.673958 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:57.673927 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fgrxl" event={"ID":"eb17e69e-d29d-4ec2-9443-2509b6b827e0","Type":"ContainerStarted","Data":"0fcf48bf2560002479937783fed7732201371d35065a3bb61e0381c7cdacda5f"} Jul 18 01:53:57.675116 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:57.675085 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nbcc6" event={"ID":"9273d20a-478a-4472-afd4-218bb890a4b6","Type":"ContainerStarted","Data":"87bdf9aac754605adb4712d268097b70e432254b37f28369d1e4cb2b5f26edb3"} Jul 18 01:53:59.686016 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:59.685977 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-fgrxl" event={"ID":"eb17e69e-d29d-4ec2-9443-2509b6b827e0","Type":"ContainerStarted","Data":"9f6f5ff081ea60728ab0f309ed90959bab918d1415415817ed45f521d1185e9a"} Jul 18 01:53:59.687434 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:59.687404 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nbcc6" event={"ID":"9273d20a-478a-4472-afd4-218bb890a4b6","Type":"ContainerStarted","Data":"8f1952e493514335ee62455039ceddeedd8d4981531c738a6ef0f89ee0ead752"} Jul 18 01:53:59.687548 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:59.687497 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-nbcc6" podUID="9273d20a-478a-4472-afd4-218bb890a4b6" containerName="registry-server" containerID="cri-o://8f1952e493514335ee62455039ceddeedd8d4981531c738a6ef0f89ee0ead752" gracePeriod=2 Jul 18 01:53:59.709012 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:59.708967 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-fgrxl" podStartSLOduration=2.209101059 podStartE2EDuration="3.708951852s" totalImagesPullingTime="1.499850793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:59.707139004 +0000 UTC m=+379.384563118" watchObservedRunningTime="2026-07-18 01:53:59.708951852 +0000 UTC m=+379.386375962" Jul 18 01:53:59.753167 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:59.753123 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-nbcc6" podStartSLOduration=1.652830722 podStartE2EDuration="3.753107345s" totalImagesPullingTime="2.100276623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:53:59.75301598 +0000 UTC m=+379.430440100" watchObservedRunningTime="2026-07-18 01:53:59.753107345 +0000 UTC m=+379.430531456" Jul 18 01:53:59.934590 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:53:59.934562 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nbcc6" Jul 18 01:54:00.043543 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:00.043451 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mtg8v\" (UniqueName: \"kubernetes.io/projected/9273d20a-478a-4472-afd4-218bb890a4b6-kube-api-access-mtg8v\") pod \"9273d20a-478a-4472-afd4-218bb890a4b6\" (UID: \"9273d20a-478a-4472-afd4-218bb890a4b6\") " Jul 18 01:54:00.045617 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:00.045581 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9273d20a-478a-4472-afd4-218bb890a4b6-kube-api-access-mtg8v" (OuterVolumeSpecName: "kube-api-access-mtg8v") pod "9273d20a-478a-4472-afd4-218bb890a4b6" (UID: "9273d20a-478a-4472-afd4-218bb890a4b6"). InnerVolumeSpecName "kube-api-access-mtg8v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:54:00.144536 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:00.144497 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtg8v\" (UniqueName: \"kubernetes.io/projected/9273d20a-478a-4472-afd4-218bb890a4b6-kube-api-access-mtg8v\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:54:00.693918 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:00.693878 2570 generic.go:356] "Generic (PLEG): container finished" podID="9273d20a-478a-4472-afd4-218bb890a4b6" containerID="8f1952e493514335ee62455039ceddeedd8d4981531c738a6ef0f89ee0ead752" exitCode=0 Jul 18 01:54:00.694397 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:00.693931 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-nbcc6" Jul 18 01:54:00.694397 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:00.693962 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nbcc6" event={"ID":"9273d20a-478a-4472-afd4-218bb890a4b6","Type":"ContainerDied","Data":"8f1952e493514335ee62455039ceddeedd8d4981531c738a6ef0f89ee0ead752"} Jul 18 01:54:00.694397 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:00.694015 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-nbcc6" event={"ID":"9273d20a-478a-4472-afd4-218bb890a4b6","Type":"ContainerDied","Data":"87bdf9aac754605adb4712d268097b70e432254b37f28369d1e4cb2b5f26edb3"} Jul 18 01:54:00.694397 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:00.694039 2570 scope.go:117] "RemoveContainer" containerID="8f1952e493514335ee62455039ceddeedd8d4981531c738a6ef0f89ee0ead752" Jul 18 01:54:00.703128 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:00.703110 2570 scope.go:117] "RemoveContainer" containerID="8f1952e493514335ee62455039ceddeedd8d4981531c738a6ef0f89ee0ead752" Jul 18 01:54:00.703375 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:00.703358 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f1952e493514335ee62455039ceddeedd8d4981531c738a6ef0f89ee0ead752\": container with ID starting with 8f1952e493514335ee62455039ceddeedd8d4981531c738a6ef0f89ee0ead752 not found: ID does not exist" containerID="8f1952e493514335ee62455039ceddeedd8d4981531c738a6ef0f89ee0ead752" Jul 18 01:54:00.703415 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:00.703383 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f1952e493514335ee62455039ceddeedd8d4981531c738a6ef0f89ee0ead752"} err="failed to get container status \"8f1952e493514335ee62455039ceddeedd8d4981531c738a6ef0f89ee0ead752\": rpc error: code = NotFound desc = could not find container \"8f1952e493514335ee62455039ceddeedd8d4981531c738a6ef0f89ee0ead752\": container with ID starting with 8f1952e493514335ee62455039ceddeedd8d4981531c738a6ef0f89ee0ead752 not found: ID does not exist" Jul 18 01:54:00.728202 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:00.728170 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nbcc6"] Jul 18 01:54:00.737075 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:00.737050 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-nbcc6"] Jul 18 01:54:00.894016 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:00.893987 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9273d20a-478a-4472-afd4-218bb890a4b6" path="/var/lib/kubelet/pods/9273d20a-478a-4472-afd4-218bb890a4b6/volumes" Jul 18 01:54:07.150870 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:07.150836 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-fgrxl" Jul 18 01:54:07.151362 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:07.150896 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-fgrxl" Jul 18 01:54:07.173423 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:07.173391 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-fgrxl" Jul 18 01:54:07.746895 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:07.746869 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-fgrxl" Jul 18 01:54:28.847831 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:28.847794 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nl9h"] Jul 18 01:54:28.848218 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:28.848130 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9273d20a-478a-4472-afd4-218bb890a4b6" containerName="registry-server" Jul 18 01:54:28.848218 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:28.848141 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9273d20a-478a-4472-afd4-218bb890a4b6" containerName="registry-server" Jul 18 01:54:28.848218 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:28.848196 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9273d20a-478a-4472-afd4-218bb890a4b6" containerName="registry-server" Jul 18 01:54:28.851264 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:28.851248 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nl9h" Jul 18 01:54:28.855210 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:28.855187 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 18 01:54:28.855587 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:28.855573 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-mw2vt\"" Jul 18 01:54:28.876238 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:28.876207 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nl9h"] Jul 18 01:54:29.003355 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:29.003315 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztdcm\" (UniqueName: \"kubernetes.io/projected/6b5b4a89-42d7-4f2d-b73a-decefd4973c2-kube-api-access-ztdcm\") pod \"dns-operator-controller-manager-864dd78c4c-4nl9h\" (UID: \"6b5b4a89-42d7-4f2d-b73a-decefd4973c2\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nl9h" Jul 18 01:54:29.104106 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:29.104018 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztdcm\" (UniqueName: \"kubernetes.io/projected/6b5b4a89-42d7-4f2d-b73a-decefd4973c2-kube-api-access-ztdcm\") pod \"dns-operator-controller-manager-864dd78c4c-4nl9h\" (UID: \"6b5b4a89-42d7-4f2d-b73a-decefd4973c2\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nl9h" Jul 18 01:54:29.120744 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:29.120713 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztdcm\" (UniqueName: \"kubernetes.io/projected/6b5b4a89-42d7-4f2d-b73a-decefd4973c2-kube-api-access-ztdcm\") pod \"dns-operator-controller-manager-864dd78c4c-4nl9h\" (UID: \"6b5b4a89-42d7-4f2d-b73a-decefd4973c2\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nl9h" Jul 18 01:54:29.160919 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:29.160878 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nl9h" Jul 18 01:54:29.333625 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:29.333600 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nl9h"] Jul 18 01:54:29.336065 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:54:29.336037 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b5b4a89_42d7_4f2d_b73a_decefd4973c2.slice/crio-af93ef003378b87e5dd25cc6a42ff679c4bce8db0dd576f7128078ab1b2d3c1e WatchSource:0}: Error finding container af93ef003378b87e5dd25cc6a42ff679c4bce8db0dd576f7128078ab1b2d3c1e: Status 404 returned error can't find the container with id af93ef003378b87e5dd25cc6a42ff679c4bce8db0dd576f7128078ab1b2d3c1e Jul 18 01:54:29.816553 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:29.816513 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nl9h" event={"ID":"6b5b4a89-42d7-4f2d-b73a-decefd4973c2","Type":"ContainerStarted","Data":"af93ef003378b87e5dd25cc6a42ff679c4bce8db0dd576f7128078ab1b2d3c1e"} Jul 18 01:54:31.828390 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:31.828355 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nl9h" event={"ID":"6b5b4a89-42d7-4f2d-b73a-decefd4973c2","Type":"ContainerStarted","Data":"e47528f04715dbad8252a4064be5dd33199dc6b8d961fb2a0cba5e370b9edb5d"} Jul 18 01:54:31.828811 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:31.828503 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nl9h" Jul 18 01:54:31.850752 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:31.850690 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nl9h" podStartSLOduration=1.923828013 podStartE2EDuration="3.850669435s" totalImagesPullingTime="1.926841422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:31.849297299 +0000 UTC m=+411.526721409" watchObservedRunningTime="2026-07-18 01:54:31.850669435 +0000 UTC m=+411.528093545" Jul 18 01:54:32.300653 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:32.300609 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr"] Jul 18 01:54:32.308762 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:32.308737 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" Jul 18 01:54:32.312178 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:32.312157 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-gtsc2\"" Jul 18 01:54:32.355073 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:32.355031 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr"] Jul 18 01:54:32.434458 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:32.434424 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6t2z\" (UniqueName: \"kubernetes.io/projected/9460feb9-5412-4a61-a5b2-222148c96ccf-kube-api-access-x6t2z\") pod \"limitador-operator-controller-manager-859d7b78bd-hvxmr\" (UID: \"9460feb9-5412-4a61-a5b2-222148c96ccf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" Jul 18 01:54:32.535008 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:32.534970 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6t2z\" (UniqueName: \"kubernetes.io/projected/9460feb9-5412-4a61-a5b2-222148c96ccf-kube-api-access-x6t2z\") pod \"limitador-operator-controller-manager-859d7b78bd-hvxmr\" (UID: \"9460feb9-5412-4a61-a5b2-222148c96ccf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" Jul 18 01:54:32.553145 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:32.553085 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6t2z\" (UniqueName: \"kubernetes.io/projected/9460feb9-5412-4a61-a5b2-222148c96ccf-kube-api-access-x6t2z\") pod \"limitador-operator-controller-manager-859d7b78bd-hvxmr\" (UID: \"9460feb9-5412-4a61-a5b2-222148c96ccf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" Jul 18 01:54:32.618539 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:32.618504 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" Jul 18 01:54:32.778149 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:32.778122 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr"] Jul 18 01:54:32.780722 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:54:32.780687 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9460feb9_5412_4a61_a5b2_222148c96ccf.slice/crio-e241af12d9e901c94ff6372a43cadaccc3e54fe4d097fbaf940e07658adc1cf5 WatchSource:0}: Error finding container e241af12d9e901c94ff6372a43cadaccc3e54fe4d097fbaf940e07658adc1cf5: Status 404 returned error can't find the container with id e241af12d9e901c94ff6372a43cadaccc3e54fe4d097fbaf940e07658adc1cf5 Jul 18 01:54:32.834980 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:32.834892 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" event={"ID":"9460feb9-5412-4a61-a5b2-222148c96ccf","Type":"ContainerStarted","Data":"e241af12d9e901c94ff6372a43cadaccc3e54fe4d097fbaf940e07658adc1cf5"} Jul 18 01:54:34.846183 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:34.846151 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" event={"ID":"9460feb9-5412-4a61-a5b2-222148c96ccf","Type":"ContainerStarted","Data":"7e5521149feb9610204341b565b786a095fa2c08a302faa54661d4cbbef32eef"} Jul 18 01:54:34.846573 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:34.846256 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" Jul 18 01:54:34.863595 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:34.863544 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" podStartSLOduration=1.243113791 podStartE2EDuration="2.8635284s" totalImagesPullingTime="1.620414609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:34.862478949 +0000 UTC m=+414.539903056" watchObservedRunningTime="2026-07-18 01:54:34.8635284 +0000 UTC m=+414.540952582" Jul 18 01:54:37.147337 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:37.147303 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6"] Jul 18 01:54:37.150772 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:37.150748 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:37.153119 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:37.153098 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4g74j\"" Jul 18 01:54:37.170367 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:37.170340 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6"] Jul 18 01:54:37.277204 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:37.277167 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/443ee306-f92e-467f-8e5d-c872b011f75a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vx9x6\" (UID: \"443ee306-f92e-467f-8e5d-c872b011f75a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:37.277373 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:37.277235 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lqrp\" (UniqueName: \"kubernetes.io/projected/443ee306-f92e-467f-8e5d-c872b011f75a-kube-api-access-9lqrp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vx9x6\" (UID: \"443ee306-f92e-467f-8e5d-c872b011f75a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:37.378662 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:37.378597 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/443ee306-f92e-467f-8e5d-c872b011f75a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vx9x6\" (UID: \"443ee306-f92e-467f-8e5d-c872b011f75a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:37.378840 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:37.378691 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lqrp\" (UniqueName: \"kubernetes.io/projected/443ee306-f92e-467f-8e5d-c872b011f75a-kube-api-access-9lqrp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vx9x6\" (UID: \"443ee306-f92e-467f-8e5d-c872b011f75a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:37.378996 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:37.378977 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/443ee306-f92e-467f-8e5d-c872b011f75a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vx9x6\" (UID: \"443ee306-f92e-467f-8e5d-c872b011f75a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:37.388884 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:37.388846 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lqrp\" (UniqueName: \"kubernetes.io/projected/443ee306-f92e-467f-8e5d-c872b011f75a-kube-api-access-9lqrp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vx9x6\" (UID: \"443ee306-f92e-467f-8e5d-c872b011f75a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:37.461728 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:37.461621 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:37.622892 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:37.622773 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6"] Jul 18 01:54:37.625630 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:54:37.625590 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod443ee306_f92e_467f_8e5d_c872b011f75a.slice/crio-24aad429cf37844efc4f1e7a275f3d50133072e202b59517f66b7a4d4bd29828 WatchSource:0}: Error finding container 24aad429cf37844efc4f1e7a275f3d50133072e202b59517f66b7a4d4bd29828: Status 404 returned error can't find the container with id 24aad429cf37844efc4f1e7a275f3d50133072e202b59517f66b7a4d4bd29828 Jul 18 01:54:37.861621 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:37.861589 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" event={"ID":"443ee306-f92e-467f-8e5d-c872b011f75a","Type":"ContainerStarted","Data":"24aad429cf37844efc4f1e7a275f3d50133072e202b59517f66b7a4d4bd29828"} Jul 18 01:54:40.456024 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:40.455985 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kgpc4"] Jul 18 01:54:40.459421 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:40.459401 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kgpc4" Jul 18 01:54:40.461876 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:40.461848 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-8m4fb\"" Jul 18 01:54:40.472163 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:40.472138 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kgpc4"] Jul 18 01:54:40.506566 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:40.506523 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4x7f\" (UniqueName: \"kubernetes.io/projected/49b16d2d-a19a-456c-9cdb-4bffe2097d5b-kube-api-access-x4x7f\") pod \"authorino-operator-75cc8d5cb7-kgpc4\" (UID: \"49b16d2d-a19a-456c-9cdb-4bffe2097d5b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kgpc4" Jul 18 01:54:40.607221 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:40.607181 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4x7f\" (UniqueName: \"kubernetes.io/projected/49b16d2d-a19a-456c-9cdb-4bffe2097d5b-kube-api-access-x4x7f\") pod \"authorino-operator-75cc8d5cb7-kgpc4\" (UID: \"49b16d2d-a19a-456c-9cdb-4bffe2097d5b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kgpc4" Jul 18 01:54:40.615996 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:40.615970 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4x7f\" (UniqueName: \"kubernetes.io/projected/49b16d2d-a19a-456c-9cdb-4bffe2097d5b-kube-api-access-x4x7f\") pod \"authorino-operator-75cc8d5cb7-kgpc4\" (UID: \"49b16d2d-a19a-456c-9cdb-4bffe2097d5b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-kgpc4" Jul 18 01:54:40.771744 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:40.771661 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-8m4fb\"" Jul 18 01:54:40.779408 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:40.779383 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kgpc4" Jul 18 01:54:40.945630 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:40.945608 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-kgpc4"] Jul 18 01:54:40.948596 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:54:40.948571 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49b16d2d_a19a_456c_9cdb_4bffe2097d5b.slice/crio-67a8a8f5bc6df9a4ca348a464bfc24e8287a739e24622d16f6a5f5b6eab7b90d WatchSource:0}: Error finding container 67a8a8f5bc6df9a4ca348a464bfc24e8287a739e24622d16f6a5f5b6eab7b90d: Status 404 returned error can't find the container with id 67a8a8f5bc6df9a4ca348a464bfc24e8287a739e24622d16f6a5f5b6eab7b90d Jul 18 01:54:41.883328 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:41.883284 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kgpc4" event={"ID":"49b16d2d-a19a-456c-9cdb-4bffe2097d5b","Type":"ContainerStarted","Data":"67a8a8f5bc6df9a4ca348a464bfc24e8287a739e24622d16f6a5f5b6eab7b90d"} Jul 18 01:54:42.838208 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:42.838177 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4nl9h" Jul 18 01:54:44.899708 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:44.899672 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kgpc4" event={"ID":"49b16d2d-a19a-456c-9cdb-4bffe2097d5b","Type":"ContainerStarted","Data":"306fa0821978cfa874adc3768ca423e28b7f5fc1d13f1b11575d85d024588353"} Jul 18 01:54:44.900151 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:44.899858 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kgpc4" Jul 18 01:54:44.901344 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:44.901316 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" event={"ID":"443ee306-f92e-467f-8e5d-c872b011f75a","Type":"ContainerStarted","Data":"b3ab3d0ba134b0f3b3a29296e3a4061fd90a7ec3b801598cafd8c4d1baf4b091"} Jul 18 01:54:44.901469 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:44.901450 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:44.918080 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:44.918026 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kgpc4" podStartSLOduration=1.402043674 podStartE2EDuration="4.918011079s" totalImagesPullingTime="3.515967405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:44.916322028 +0000 UTC m=+424.593746140" watchObservedRunningTime="2026-07-18 01:54:44.918011079 +0000 UTC m=+424.595435184" Jul 18 01:54:44.936801 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:44.936746 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" podStartSLOduration=1.087369391 podStartE2EDuration="7.936727s" totalImagesPullingTime="6.849357609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:44.935021134 +0000 UTC m=+424.612445267" watchObservedRunningTime="2026-07-18 01:54:44.936727 +0000 UTC m=+424.614151113" Jul 18 01:54:45.853010 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:45.852979 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" Jul 18 01:54:52.493052 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:52.493016 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b55dbc779-c9wqq"] Jul 18 01:54:55.907684 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:55.907626 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-kgpc4" Jul 18 01:54:55.908141 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:55.907796 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:56.953825 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:56.953794 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5"] Jul 18 01:54:56.957421 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:56.957401 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:54:56.969474 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:56.969448 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5"] Jul 18 01:54:57.051575 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.051532 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f000024-f15f-4f30-9a27-58db3838d48c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qqwm5\" (UID: \"0f000024-f15f-4f30-9a27-58db3838d48c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:54:57.051782 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.051623 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfmf4\" (UniqueName: \"kubernetes.io/projected/0f000024-f15f-4f30-9a27-58db3838d48c-kube-api-access-gfmf4\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qqwm5\" (UID: \"0f000024-f15f-4f30-9a27-58db3838d48c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:54:57.152124 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.152090 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfmf4\" (UniqueName: \"kubernetes.io/projected/0f000024-f15f-4f30-9a27-58db3838d48c-kube-api-access-gfmf4\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qqwm5\" (UID: \"0f000024-f15f-4f30-9a27-58db3838d48c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:54:57.152304 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.152161 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f000024-f15f-4f30-9a27-58db3838d48c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qqwm5\" (UID: \"0f000024-f15f-4f30-9a27-58db3838d48c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:54:57.152567 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.152545 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f000024-f15f-4f30-9a27-58db3838d48c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qqwm5\" (UID: \"0f000024-f15f-4f30-9a27-58db3838d48c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:54:57.161315 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.161289 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfmf4\" (UniqueName: \"kubernetes.io/projected/0f000024-f15f-4f30-9a27-58db3838d48c-kube-api-access-gfmf4\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-qqwm5\" (UID: \"0f000024-f15f-4f30-9a27-58db3838d48c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:54:57.267888 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.267792 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:54:57.409082 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.404952 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5"] Jul 18 01:54:57.634087 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.634042 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5"] Jul 18 01:54:57.636038 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.636008 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5"] Jul 18 01:54:57.649808 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.649780 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6"] Jul 18 01:54:57.650055 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.650013 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" podUID="443ee306-f92e-467f-8e5d-c872b011f75a" containerName="manager" containerID="cri-o://b3ab3d0ba134b0f3b3a29296e3a4061fd90a7ec3b801598cafd8c4d1baf4b091" gracePeriod=2 Jul 18 01:54:57.656682 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.656655 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv"] Jul 18 01:54:57.657179 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.657160 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f000024-f15f-4f30-9a27-58db3838d48c" containerName="manager" Jul 18 01:54:57.657179 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.657179 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f000024-f15f-4f30-9a27-58db3838d48c" containerName="manager" Jul 18 01:54:57.657339 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.657279 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f000024-f15f-4f30-9a27-58db3838d48c" containerName="manager" Jul 18 01:54:57.660760 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.660739 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" Jul 18 01:54:57.665890 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.665867 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6"] Jul 18 01:54:57.670161 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.670097 2570 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"443ee306-f92e-467f-8e5d-c872b011f75a\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-vx9x6\": pods \"kuadrant-operator-controller-manager-8684c9ff49-vx9x6\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:57.672357 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.672322 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vx9x6\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="443ee306-f92e-467f-8e5d-c872b011f75a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:57.672979 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.672958 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv"] Jul 18 01:54:57.683496 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.683472 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr"] Jul 18 01:54:57.683727 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.683703 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" podUID="9460feb9-5412-4a61-a5b2-222148c96ccf" containerName="manager" containerID="cri-o://7e5521149feb9610204341b565b786a095fa2c08a302faa54661d4cbbef32eef" gracePeriod=2 Jul 18 01:54:57.690167 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.690139 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr"] Jul 18 01:54:57.699860 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.699828 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vx9x6\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="443ee306-f92e-467f-8e5d-c872b011f75a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:57.705633 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.705608 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hvxmr\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="9460feb9-5412-4a61-a5b2-222148c96ccf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" Jul 18 01:54:57.714898 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.714872 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7zv"] Jul 18 01:54:57.715391 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.715375 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9460feb9-5412-4a61-a5b2-222148c96ccf" containerName="manager" Jul 18 01:54:57.715435 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.715394 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9460feb9-5412-4a61-a5b2-222148c96ccf" containerName="manager" Jul 18 01:54:57.715435 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.715410 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="443ee306-f92e-467f-8e5d-c872b011f75a" containerName="manager" Jul 18 01:54:57.715435 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.715419 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="443ee306-f92e-467f-8e5d-c872b011f75a" containerName="manager" Jul 18 01:54:57.715539 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.715516 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9460feb9-5412-4a61-a5b2-222148c96ccf" containerName="manager" Jul 18 01:54:57.715539 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.715529 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="443ee306-f92e-467f-8e5d-c872b011f75a" containerName="manager" Jul 18 01:54:57.718907 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.718889 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7zv" Jul 18 01:54:57.721276 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.721248 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hvxmr\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="9460feb9-5412-4a61-a5b2-222148c96ccf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" Jul 18 01:54:57.729390 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.729345 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7zv"] Jul 18 01:54:57.747028 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.746990 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vx9x6\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="443ee306-f92e-467f-8e5d-c872b011f75a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:57.757954 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.757927 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hzgg\" (UniqueName: \"kubernetes.io/projected/22680e42-703c-483d-8bf3-ab25b40c43c4-kube-api-access-6hzgg\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-f7lzv\" (UID: \"22680e42-703c-483d-8bf3-ab25b40c43c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" Jul 18 01:54:57.758095 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.757989 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrvbt\" (UniqueName: \"kubernetes.io/projected/56774121-f7ed-4af6-9179-e0203a081e7e-kube-api-access-zrvbt\") pod \"limitador-operator-controller-manager-859d7b78bd-vq7zv\" (UID: \"56774121-f7ed-4af6-9179-e0203a081e7e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7zv" Jul 18 01:54:57.758141 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.758094 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/22680e42-703c-483d-8bf3-ab25b40c43c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-f7lzv\" (UID: \"22680e42-703c-483d-8bf3-ab25b40c43c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" Jul 18 01:54:57.859353 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.859313 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/22680e42-703c-483d-8bf3-ab25b40c43c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-f7lzv\" (UID: \"22680e42-703c-483d-8bf3-ab25b40c43c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" Jul 18 01:54:57.859491 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.859397 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hzgg\" (UniqueName: \"kubernetes.io/projected/22680e42-703c-483d-8bf3-ab25b40c43c4-kube-api-access-6hzgg\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-f7lzv\" (UID: \"22680e42-703c-483d-8bf3-ab25b40c43c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" Jul 18 01:54:57.859544 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.859488 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrvbt\" (UniqueName: \"kubernetes.io/projected/56774121-f7ed-4af6-9179-e0203a081e7e-kube-api-access-zrvbt\") pod \"limitador-operator-controller-manager-859d7b78bd-vq7zv\" (UID: \"56774121-f7ed-4af6-9179-e0203a081e7e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7zv" Jul 18 01:54:57.859709 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.859689 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/22680e42-703c-483d-8bf3-ab25b40c43c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-f7lzv\" (UID: \"22680e42-703c-483d-8bf3-ab25b40c43c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" Jul 18 01:54:57.869304 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.869252 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrvbt\" (UniqueName: \"kubernetes.io/projected/56774121-f7ed-4af6-9179-e0203a081e7e-kube-api-access-zrvbt\") pod \"limitador-operator-controller-manager-859d7b78bd-vq7zv\" (UID: \"56774121-f7ed-4af6-9179-e0203a081e7e\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7zv" Jul 18 01:54:57.869696 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.869672 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hzgg\" (UniqueName: \"kubernetes.io/projected/22680e42-703c-483d-8bf3-ab25b40c43c4-kube-api-access-6hzgg\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-f7lzv\" (UID: \"22680e42-703c-483d-8bf3-ab25b40c43c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" Jul 18 01:54:57.914139 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.914109 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:57.916788 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.916758 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vx9x6\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="443ee306-f92e-467f-8e5d-c872b011f75a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:57.919142 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.919115 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hvxmr\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="9460feb9-5412-4a61-a5b2-222148c96ccf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" Jul 18 01:54:57.922846 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.922827 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" Jul 18 01:54:57.924771 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.924747 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vx9x6\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="443ee306-f92e-467f-8e5d-c872b011f75a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:57.928693 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.928674 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hvxmr\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="9460feb9-5412-4a61-a5b2-222148c96ccf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" Jul 18 01:54:57.960524 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.960505 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/443ee306-f92e-467f-8e5d-c872b011f75a-extensions-socket-volume\") pod \"443ee306-f92e-467f-8e5d-c872b011f75a\" (UID: \"443ee306-f92e-467f-8e5d-c872b011f75a\") " Jul 18 01:54:57.960953 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.960635 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lqrp\" (UniqueName: \"kubernetes.io/projected/443ee306-f92e-467f-8e5d-c872b011f75a-kube-api-access-9lqrp\") pod \"443ee306-f92e-467f-8e5d-c872b011f75a\" (UID: \"443ee306-f92e-467f-8e5d-c872b011f75a\") " Jul 18 01:54:57.961070 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.961044 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/443ee306-f92e-467f-8e5d-c872b011f75a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "443ee306-f92e-467f-8e5d-c872b011f75a" (UID: "443ee306-f92e-467f-8e5d-c872b011f75a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:54:57.962247 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.962227 2570 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" container="manager" Jul 18 01:54:57.962985 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.962964 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/443ee306-f92e-467f-8e5d-c872b011f75a-kube-api-access-9lqrp" (OuterVolumeSpecName: "kube-api-access-9lqrp") pod "443ee306-f92e-467f-8e5d-c872b011f75a" (UID: "443ee306-f92e-467f-8e5d-c872b011f75a"). InnerVolumeSpecName "kube-api-access-9lqrp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:54:57.963426 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.963402 2570 generic.go:356] "Generic (PLEG): container finished" podID="443ee306-f92e-467f-8e5d-c872b011f75a" containerID="b3ab3d0ba134b0f3b3a29296e3a4061fd90a7ec3b801598cafd8c4d1baf4b091" exitCode=0 Jul 18 01:54:57.963511 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.963445 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:57.963570 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.963513 2570 scope.go:117] "RemoveContainer" containerID="b3ab3d0ba134b0f3b3a29296e3a4061fd90a7ec3b801598cafd8c4d1baf4b091" Jul 18 01:54:57.964271 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.964243 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-qqwm5\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="0f000024-f15f-4f30-9a27-58db3838d48c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:54:57.965021 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.964995 2570 generic.go:356] "Generic (PLEG): container finished" podID="9460feb9-5412-4a61-a5b2-222148c96ccf" containerID="7e5521149feb9610204341b565b786a095fa2c08a302faa54661d4cbbef32eef" exitCode=0 Jul 18 01:54:57.965130 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.965032 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" Jul 18 01:54:57.966187 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.966164 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vx9x6\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="443ee306-f92e-467f-8e5d-c872b011f75a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:57.968001 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.967979 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hvxmr\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="9460feb9-5412-4a61-a5b2-222148c96ccf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" Jul 18 01:54:57.969787 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.969759 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hvxmr\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="9460feb9-5412-4a61-a5b2-222148c96ccf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" Jul 18 01:54:57.971508 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.971483 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-qqwm5\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="0f000024-f15f-4f30-9a27-58db3838d48c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:54:57.973131 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.973116 2570 scope.go:117] "RemoveContainer" containerID="b3ab3d0ba134b0f3b3a29296e3a4061fd90a7ec3b801598cafd8c4d1baf4b091" Jul 18 01:54:57.973229 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.973188 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vx9x6\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="443ee306-f92e-467f-8e5d-c872b011f75a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:57.973377 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.973353 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3ab3d0ba134b0f3b3a29296e3a4061fd90a7ec3b801598cafd8c4d1baf4b091\": container with ID starting with b3ab3d0ba134b0f3b3a29296e3a4061fd90a7ec3b801598cafd8c4d1baf4b091 not found: ID does not exist" containerID="b3ab3d0ba134b0f3b3a29296e3a4061fd90a7ec3b801598cafd8c4d1baf4b091" Jul 18 01:54:57.973420 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.973386 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3ab3d0ba134b0f3b3a29296e3a4061fd90a7ec3b801598cafd8c4d1baf4b091"} err="failed to get container status \"b3ab3d0ba134b0f3b3a29296e3a4061fd90a7ec3b801598cafd8c4d1baf4b091\": rpc error: code = NotFound desc = could not find container \"b3ab3d0ba134b0f3b3a29296e3a4061fd90a7ec3b801598cafd8c4d1baf4b091\": container with ID starting with b3ab3d0ba134b0f3b3a29296e3a4061fd90a7ec3b801598cafd8c4d1baf4b091 not found: ID does not exist" Jul 18 01:54:57.973420 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.973403 2570 scope.go:117] "RemoveContainer" containerID="7e5521149feb9610204341b565b786a095fa2c08a302faa54661d4cbbef32eef" Jul 18 01:54:57.974939 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.974917 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-qqwm5\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="0f000024-f15f-4f30-9a27-58db3838d48c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:54:57.976720 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.976695 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vx9x6\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="443ee306-f92e-467f-8e5d-c872b011f75a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:57.978337 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.978315 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hvxmr\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="9460feb9-5412-4a61-a5b2-222148c96ccf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" Jul 18 01:54:57.982254 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.982236 2570 scope.go:117] "RemoveContainer" containerID="7e5521149feb9610204341b565b786a095fa2c08a302faa54661d4cbbef32eef" Jul 18 01:54:57.982594 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:57.982498 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e5521149feb9610204341b565b786a095fa2c08a302faa54661d4cbbef32eef\": container with ID starting with 7e5521149feb9610204341b565b786a095fa2c08a302faa54661d4cbbef32eef not found: ID does not exist" containerID="7e5521149feb9610204341b565b786a095fa2c08a302faa54661d4cbbef32eef" Jul 18 01:54:57.982594 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:57.982520 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e5521149feb9610204341b565b786a095fa2c08a302faa54661d4cbbef32eef"} err="failed to get container status \"7e5521149feb9610204341b565b786a095fa2c08a302faa54661d4cbbef32eef\": rpc error: code = NotFound desc = could not find container \"7e5521149feb9610204341b565b786a095fa2c08a302faa54661d4cbbef32eef\": container with ID starting with 7e5521149feb9610204341b565b786a095fa2c08a302faa54661d4cbbef32eef not found: ID does not exist" Jul 18 01:54:58.061756 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.061724 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x6t2z\" (UniqueName: \"kubernetes.io/projected/9460feb9-5412-4a61-a5b2-222148c96ccf-kube-api-access-x6t2z\") pod \"9460feb9-5412-4a61-a5b2-222148c96ccf\" (UID: \"9460feb9-5412-4a61-a5b2-222148c96ccf\") " Jul 18 01:54:58.061912 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.061878 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lqrp\" (UniqueName: \"kubernetes.io/projected/443ee306-f92e-467f-8e5d-c872b011f75a-kube-api-access-9lqrp\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:54:58.061912 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.061888 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/443ee306-f92e-467f-8e5d-c872b011f75a-extensions-socket-volume\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:54:58.063753 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.063726 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9460feb9-5412-4a61-a5b2-222148c96ccf-kube-api-access-x6t2z" (OuterVolumeSpecName: "kube-api-access-x6t2z") pod "9460feb9-5412-4a61-a5b2-222148c96ccf" (UID: "9460feb9-5412-4a61-a5b2-222148c96ccf"). InnerVolumeSpecName "kube-api-access-x6t2z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:54:58.065821 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.065799 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" Jul 18 01:54:58.085411 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.085388 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7zv" Jul 18 01:54:58.162614 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.162408 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6t2z\" (UniqueName: \"kubernetes.io/projected/9460feb9-5412-4a61-a5b2-222148c96ccf-kube-api-access-x6t2z\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:54:58.241208 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.241185 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv"] Jul 18 01:54:58.243081 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:54:58.243050 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22680e42_703c_483d_8bf3_ab25b40c43c4.slice/crio-baf6d595ff81c6e88035228a5b5f9db0a31dd18d6ef8b81bc0d757468fa5a905 WatchSource:0}: Error finding container baf6d595ff81c6e88035228a5b5f9db0a31dd18d6ef8b81bc0d757468fa5a905: Status 404 returned error can't find the container with id baf6d595ff81c6e88035228a5b5f9db0a31dd18d6ef8b81bc0d757468fa5a905 Jul 18 01:54:58.263433 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.263387 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7zv"] Jul 18 01:54:58.266265 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:54:58.266237 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56774121_f7ed_4af6_9179_e0203a081e7e.slice/crio-208f88bd2eecf72956c5b93ef7635762a698412e9cb4a46f7ee2679e592a9c14 WatchSource:0}: Error finding container 208f88bd2eecf72956c5b93ef7635762a698412e9cb4a46f7ee2679e592a9c14: Status 404 returned error can't find the container with id 208f88bd2eecf72956c5b93ef7635762a698412e9cb4a46f7ee2679e592a9c14 Jul 18 01:54:58.305104 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:58.305074 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-qqwm5\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="0f000024-f15f-4f30-9a27-58db3838d48c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:54:58.307279 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:58.307254 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vx9x6\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="443ee306-f92e-467f-8e5d-c872b011f75a" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vx9x6" Jul 18 01:54:58.309099 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:58.309075 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-hvxmr\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="9460feb9-5412-4a61-a5b2-222148c96ccf" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hvxmr" Jul 18 01:54:58.894431 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.894396 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="443ee306-f92e-467f-8e5d-c872b011f75a" path="/var/lib/kubelet/pods/443ee306-f92e-467f-8e5d-c872b011f75a/volumes" Jul 18 01:54:58.894760 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.894747 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9460feb9-5412-4a61-a5b2-222148c96ccf" path="/var/lib/kubelet/pods/9460feb9-5412-4a61-a5b2-222148c96ccf/volumes" Jul 18 01:54:58.975106 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.975066 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" event={"ID":"22680e42-703c-483d-8bf3-ab25b40c43c4","Type":"ContainerStarted","Data":"6a028e3a0a788007394e95dd9575adb3f2533685eecb34fa4135f5b546e927b4"} Jul 18 01:54:58.975573 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.975127 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" event={"ID":"22680e42-703c-483d-8bf3-ab25b40c43c4","Type":"ContainerStarted","Data":"baf6d595ff81c6e88035228a5b5f9db0a31dd18d6ef8b81bc0d757468fa5a905"} Jul 18 01:54:58.975573 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.975243 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" Jul 18 01:54:58.976835 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.976805 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7zv" event={"ID":"56774121-f7ed-4af6-9179-e0203a081e7e","Type":"ContainerStarted","Data":"c32aef7221f5d37dfda7a26583dc21bac51aea7715e589e6497384a824a14ea4"} Jul 18 01:54:58.976954 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.976850 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7zv" event={"ID":"56774121-f7ed-4af6-9179-e0203a081e7e","Type":"ContainerStarted","Data":"208f88bd2eecf72956c5b93ef7635762a698412e9cb4a46f7ee2679e592a9c14"} Jul 18 01:54:58.976954 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.976898 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7zv" Jul 18 01:54:58.978680 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.978654 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" podUID="0f000024-f15f-4f30-9a27-58db3838d48c" containerName="manager" containerID="cri-o://952b35c2ec10f8f2c093dbc397a957d38b0d4d149d9d8023308d96a873b354c3" gracePeriod=2 Jul 18 01:54:58.995258 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:58.995216 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" podStartSLOduration=1.995202145 podStartE2EDuration="1.995202145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:58.993866828 +0000 UTC m=+438.671290947" watchObservedRunningTime="2026-07-18 01:54:58.995202145 +0000 UTC m=+438.672626256" Jul 18 01:54:59.010799 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:59.010753 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7zv" podStartSLOduration=2.010733949 podStartE2EDuration="2.010733949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:59.009695542 +0000 UTC m=+438.687119654" watchObservedRunningTime="2026-07-18 01:54:59.010733949 +0000 UTC m=+438.688158059" Jul 18 01:54:59.221714 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:59.221691 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:54:59.223966 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:59.223931 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-qqwm5\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="0f000024-f15f-4f30-9a27-58db3838d48c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:54:59.272088 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:59.272047 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f000024-f15f-4f30-9a27-58db3838d48c-extensions-socket-volume\") pod \"0f000024-f15f-4f30-9a27-58db3838d48c\" (UID: \"0f000024-f15f-4f30-9a27-58db3838d48c\") " Jul 18 01:54:59.272256 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:59.272104 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfmf4\" (UniqueName: \"kubernetes.io/projected/0f000024-f15f-4f30-9a27-58db3838d48c-kube-api-access-gfmf4\") pod \"0f000024-f15f-4f30-9a27-58db3838d48c\" (UID: \"0f000024-f15f-4f30-9a27-58db3838d48c\") " Jul 18 01:54:59.272363 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:59.272330 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f000024-f15f-4f30-9a27-58db3838d48c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0f000024-f15f-4f30-9a27-58db3838d48c" (UID: "0f000024-f15f-4f30-9a27-58db3838d48c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:54:59.274210 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:59.274189 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f000024-f15f-4f30-9a27-58db3838d48c-kube-api-access-gfmf4" (OuterVolumeSpecName: "kube-api-access-gfmf4") pod "0f000024-f15f-4f30-9a27-58db3838d48c" (UID: "0f000024-f15f-4f30-9a27-58db3838d48c"). InnerVolumeSpecName "kube-api-access-gfmf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:54:59.373170 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:59.373133 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0f000024-f15f-4f30-9a27-58db3838d48c-extensions-socket-volume\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:54:59.373170 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:59.373162 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfmf4\" (UniqueName: \"kubernetes.io/projected/0f000024-f15f-4f30-9a27-58db3838d48c-kube-api-access-gfmf4\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:54:59.984382 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:59.984346 2570 generic.go:356] "Generic (PLEG): container finished" podID="0f000024-f15f-4f30-9a27-58db3838d48c" containerID="952b35c2ec10f8f2c093dbc397a957d38b0d4d149d9d8023308d96a873b354c3" exitCode=2 Jul 18 01:54:59.984819 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:59.984392 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:54:59.984819 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:59.984441 2570 scope.go:117] "RemoveContainer" containerID="952b35c2ec10f8f2c093dbc397a957d38b0d4d149d9d8023308d96a873b354c3" Jul 18 01:54:59.987225 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:59.987187 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-qqwm5\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="0f000024-f15f-4f30-9a27-58db3838d48c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:54:59.997759 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:59.997731 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-qqwm5\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="0f000024-f15f-4f30-9a27-58db3838d48c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:54:59.998869 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:59.998851 2570 scope.go:117] "RemoveContainer" containerID="952b35c2ec10f8f2c093dbc397a957d38b0d4d149d9d8023308d96a873b354c3" Jul 18 01:54:59.999139 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:54:59.999120 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"952b35c2ec10f8f2c093dbc397a957d38b0d4d149d9d8023308d96a873b354c3\": container with ID starting with 952b35c2ec10f8f2c093dbc397a957d38b0d4d149d9d8023308d96a873b354c3 not found: ID does not exist" containerID="952b35c2ec10f8f2c093dbc397a957d38b0d4d149d9d8023308d96a873b354c3" Jul 18 01:54:59.999198 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:54:59.999151 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"952b35c2ec10f8f2c093dbc397a957d38b0d4d149d9d8023308d96a873b354c3"} err="failed to get container status \"952b35c2ec10f8f2c093dbc397a957d38b0d4d149d9d8023308d96a873b354c3\": rpc error: code = NotFound desc = could not find container \"952b35c2ec10f8f2c093dbc397a957d38b0d4d149d9d8023308d96a873b354c3\": container with ID starting with 952b35c2ec10f8f2c093dbc397a957d38b0d4d149d9d8023308d96a873b354c3 not found: ID does not exist" Jul 18 01:55:00.892846 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:55:00.892805 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-qqwm5\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="0f000024-f15f-4f30-9a27-58db3838d48c" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-qqwm5" Jul 18 01:55:00.895342 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:00.895315 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f000024-f15f-4f30-9a27-58db3838d48c" path="/var/lib/kubelet/pods/0f000024-f15f-4f30-9a27-58db3838d48c/volumes" Jul 18 01:55:09.986756 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:09.986681 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" Jul 18 01:55:09.986756 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:09.986754 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-vq7zv" Jul 18 01:55:10.282181 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:10.282094 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx"] Jul 18 01:55:10.287272 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:10.287248 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" Jul 18 01:55:10.301275 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:10.301251 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx"] Jul 18 01:55:10.366568 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:10.366530 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbwcc\" (UniqueName: \"kubernetes.io/projected/aa5e9117-9d90-4255-a56a-52b9cf3b816b-kube-api-access-tbwcc\") pod \"kuadrant-operator-controller-manager-86878c769f-fjxwx\" (UID: \"aa5e9117-9d90-4255-a56a-52b9cf3b816b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" Jul 18 01:55:10.366770 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:10.366661 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa5e9117-9d90-4255-a56a-52b9cf3b816b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fjxwx\" (UID: \"aa5e9117-9d90-4255-a56a-52b9cf3b816b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" Jul 18 01:55:10.467792 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:10.467756 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa5e9117-9d90-4255-a56a-52b9cf3b816b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fjxwx\" (UID: \"aa5e9117-9d90-4255-a56a-52b9cf3b816b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" Jul 18 01:55:10.467966 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:10.467822 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbwcc\" (UniqueName: \"kubernetes.io/projected/aa5e9117-9d90-4255-a56a-52b9cf3b816b-kube-api-access-tbwcc\") pod \"kuadrant-operator-controller-manager-86878c769f-fjxwx\" (UID: \"aa5e9117-9d90-4255-a56a-52b9cf3b816b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" Jul 18 01:55:10.468230 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:10.468207 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa5e9117-9d90-4255-a56a-52b9cf3b816b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fjxwx\" (UID: \"aa5e9117-9d90-4255-a56a-52b9cf3b816b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" Jul 18 01:55:10.480995 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:10.480971 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbwcc\" (UniqueName: \"kubernetes.io/projected/aa5e9117-9d90-4255-a56a-52b9cf3b816b-kube-api-access-tbwcc\") pod \"kuadrant-operator-controller-manager-86878c769f-fjxwx\" (UID: \"aa5e9117-9d90-4255-a56a-52b9cf3b816b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" Jul 18 01:55:10.597898 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:10.597855 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" Jul 18 01:55:10.737165 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:10.737139 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx"] Jul 18 01:55:10.739427 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:55:10.739400 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa5e9117_9d90_4255_a56a_52b9cf3b816b.slice/crio-1ce8e947e983c42bb8dd135495a90176ac2a2e1d8cfdeaed252b84d0d244c6a6 WatchSource:0}: Error finding container 1ce8e947e983c42bb8dd135495a90176ac2a2e1d8cfdeaed252b84d0d244c6a6: Status 404 returned error can't find the container with id 1ce8e947e983c42bb8dd135495a90176ac2a2e1d8cfdeaed252b84d0d244c6a6 Jul 18 01:55:11.043078 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:11.043006 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" event={"ID":"aa5e9117-9d90-4255-a56a-52b9cf3b816b","Type":"ContainerStarted","Data":"3409dc81ecc0aff753be438e36d10947ca4314958f20b8cd0f782c14d63b28b3"} Jul 18 01:55:11.043078 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:11.043056 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" event={"ID":"aa5e9117-9d90-4255-a56a-52b9cf3b816b","Type":"ContainerStarted","Data":"1ce8e947e983c42bb8dd135495a90176ac2a2e1d8cfdeaed252b84d0d244c6a6"} Jul 18 01:55:11.043466 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:11.043101 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" Jul 18 01:55:11.064924 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:11.064876 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" podStartSLOduration=1.064861369 podStartE2EDuration="1.064861369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:11.062747212 +0000 UTC m=+450.740171323" watchObservedRunningTime="2026-07-18 01:55:11.064861369 +0000 UTC m=+450.742285479" Jul 18 01:55:17.515319 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.515246 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b55dbc779-c9wqq" podUID="9a86969b-dcb5-4cf7-8007-dd9ca721b6df" containerName="console" containerID="cri-o://958b997d23b43a03c82115ca1e4aee6e661d0a449c5a6f92e305ea5cff93844d" gracePeriod=15 Jul 18 01:55:17.762277 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.762255 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:55:17.841407 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.841374 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-service-ca\") pod \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " Jul 18 01:55:17.841584 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.841460 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-trusted-ca-bundle\") pod \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " Jul 18 01:55:17.841584 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.841478 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-config\") pod \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " Jul 18 01:55:17.841584 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.841496 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-serving-cert\") pod \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " Jul 18 01:55:17.841584 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.841517 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-oauth-serving-cert\") pod \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " Jul 18 01:55:17.841584 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.841548 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ws6mj\" (UniqueName: \"kubernetes.io/projected/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-kube-api-access-ws6mj\") pod \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " Jul 18 01:55:17.841584 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.841572 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-oauth-config\") pod \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\" (UID: \"9a86969b-dcb5-4cf7-8007-dd9ca721b6df\") " Jul 18 01:55:17.841925 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.841903 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-service-ca" (OuterVolumeSpecName: "service-ca") pod "9a86969b-dcb5-4cf7-8007-dd9ca721b6df" (UID: "9a86969b-dcb5-4cf7-8007-dd9ca721b6df"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:55:17.841979 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.841953 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-config" (OuterVolumeSpecName: "console-config") pod "9a86969b-dcb5-4cf7-8007-dd9ca721b6df" (UID: "9a86969b-dcb5-4cf7-8007-dd9ca721b6df"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:55:17.842061 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.842021 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9a86969b-dcb5-4cf7-8007-dd9ca721b6df" (UID: "9a86969b-dcb5-4cf7-8007-dd9ca721b6df"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:55:17.842061 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.842043 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9a86969b-dcb5-4cf7-8007-dd9ca721b6df" (UID: "9a86969b-dcb5-4cf7-8007-dd9ca721b6df"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:55:17.843809 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.843786 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9a86969b-dcb5-4cf7-8007-dd9ca721b6df" (UID: "9a86969b-dcb5-4cf7-8007-dd9ca721b6df"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:55:17.843915 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.843899 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9a86969b-dcb5-4cf7-8007-dd9ca721b6df" (UID: "9a86969b-dcb5-4cf7-8007-dd9ca721b6df"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:55:17.843993 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.843978 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-kube-api-access-ws6mj" (OuterVolumeSpecName: "kube-api-access-ws6mj") pod "9a86969b-dcb5-4cf7-8007-dd9ca721b6df" (UID: "9a86969b-dcb5-4cf7-8007-dd9ca721b6df"). InnerVolumeSpecName "kube-api-access-ws6mj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:55:17.942821 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.942783 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-oauth-config\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:55:17.942821 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.942814 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-service-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:55:17.942821 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.942824 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-trusted-ca-bundle\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:55:17.943052 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.942833 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-config\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:55:17.943052 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.942842 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-console-serving-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:55:17.943052 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.942850 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-oauth-serving-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:55:17.943052 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:17.942859 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ws6mj\" (UniqueName: \"kubernetes.io/projected/9a86969b-dcb5-4cf7-8007-dd9ca721b6df-kube-api-access-ws6mj\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:55:18.076164 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:18.076129 2570 generic.go:356] "Generic (PLEG): container finished" podID="9a86969b-dcb5-4cf7-8007-dd9ca721b6df" containerID="958b997d23b43a03c82115ca1e4aee6e661d0a449c5a6f92e305ea5cff93844d" exitCode=0 Jul 18 01:55:18.076342 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:18.076172 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b55dbc779-c9wqq" Jul 18 01:55:18.076342 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:18.076219 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b55dbc779-c9wqq" event={"ID":"9a86969b-dcb5-4cf7-8007-dd9ca721b6df","Type":"ContainerDied","Data":"958b997d23b43a03c82115ca1e4aee6e661d0a449c5a6f92e305ea5cff93844d"} Jul 18 01:55:18.076342 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:18.076268 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b55dbc779-c9wqq" event={"ID":"9a86969b-dcb5-4cf7-8007-dd9ca721b6df","Type":"ContainerDied","Data":"fe641a2ef41ef4755d91aad1c797383f127513d082d3c4cef2d17680c668321f"} Jul 18 01:55:18.076342 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:18.076289 2570 scope.go:117] "RemoveContainer" containerID="958b997d23b43a03c82115ca1e4aee6e661d0a449c5a6f92e305ea5cff93844d" Jul 18 01:55:18.085462 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:18.085439 2570 scope.go:117] "RemoveContainer" containerID="958b997d23b43a03c82115ca1e4aee6e661d0a449c5a6f92e305ea5cff93844d" Jul 18 01:55:18.085777 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:55:18.085750 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"958b997d23b43a03c82115ca1e4aee6e661d0a449c5a6f92e305ea5cff93844d\": container with ID starting with 958b997d23b43a03c82115ca1e4aee6e661d0a449c5a6f92e305ea5cff93844d not found: ID does not exist" containerID="958b997d23b43a03c82115ca1e4aee6e661d0a449c5a6f92e305ea5cff93844d" Jul 18 01:55:18.085898 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:18.085785 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"958b997d23b43a03c82115ca1e4aee6e661d0a449c5a6f92e305ea5cff93844d"} err="failed to get container status \"958b997d23b43a03c82115ca1e4aee6e661d0a449c5a6f92e305ea5cff93844d\": rpc error: code = NotFound desc = could not find container \"958b997d23b43a03c82115ca1e4aee6e661d0a449c5a6f92e305ea5cff93844d\": container with ID starting with 958b997d23b43a03c82115ca1e4aee6e661d0a449c5a6f92e305ea5cff93844d not found: ID does not exist" Jul 18 01:55:18.096393 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:18.096323 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b55dbc779-c9wqq"] Jul 18 01:55:18.101784 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:18.101761 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b55dbc779-c9wqq"] Jul 18 01:55:18.894487 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:18.894454 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a86969b-dcb5-4cf7-8007-dd9ca721b6df" path="/var/lib/kubelet/pods/9a86969b-dcb5-4cf7-8007-dd9ca721b6df/volumes" Jul 18 01:55:22.050551 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:22.050515 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" Jul 18 01:55:22.112314 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:22.112284 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv"] Jul 18 01:55:22.112509 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:22.112485 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" podUID="22680e42-703c-483d-8bf3-ab25b40c43c4" containerName="manager" containerID="cri-o://6a028e3a0a788007394e95dd9575adb3f2533685eecb34fa4135f5b546e927b4" gracePeriod=10 Jul 18 01:55:22.366524 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:22.366502 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" Jul 18 01:55:22.481584 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:22.481553 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/22680e42-703c-483d-8bf3-ab25b40c43c4-extensions-socket-volume\") pod \"22680e42-703c-483d-8bf3-ab25b40c43c4\" (UID: \"22680e42-703c-483d-8bf3-ab25b40c43c4\") " Jul 18 01:55:22.481809 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:22.481703 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hzgg\" (UniqueName: \"kubernetes.io/projected/22680e42-703c-483d-8bf3-ab25b40c43c4-kube-api-access-6hzgg\") pod \"22680e42-703c-483d-8bf3-ab25b40c43c4\" (UID: \"22680e42-703c-483d-8bf3-ab25b40c43c4\") " Jul 18 01:55:22.481913 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:22.481887 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22680e42-703c-483d-8bf3-ab25b40c43c4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "22680e42-703c-483d-8bf3-ab25b40c43c4" (UID: "22680e42-703c-483d-8bf3-ab25b40c43c4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:55:22.483839 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:22.483809 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22680e42-703c-483d-8bf3-ab25b40c43c4-kube-api-access-6hzgg" (OuterVolumeSpecName: "kube-api-access-6hzgg") pod "22680e42-703c-483d-8bf3-ab25b40c43c4" (UID: "22680e42-703c-483d-8bf3-ab25b40c43c4"). InnerVolumeSpecName "kube-api-access-6hzgg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:55:22.583373 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:22.583330 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hzgg\" (UniqueName: \"kubernetes.io/projected/22680e42-703c-483d-8bf3-ab25b40c43c4-kube-api-access-6hzgg\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:55:22.583373 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:22.583370 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/22680e42-703c-483d-8bf3-ab25b40c43c4-extensions-socket-volume\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:55:23.103859 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:23.103814 2570 generic.go:356] "Generic (PLEG): container finished" podID="22680e42-703c-483d-8bf3-ab25b40c43c4" containerID="6a028e3a0a788007394e95dd9575adb3f2533685eecb34fa4135f5b546e927b4" exitCode=0 Jul 18 01:55:23.104291 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:23.103866 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" event={"ID":"22680e42-703c-483d-8bf3-ab25b40c43c4","Type":"ContainerDied","Data":"6a028e3a0a788007394e95dd9575adb3f2533685eecb34fa4135f5b546e927b4"} Jul 18 01:55:23.104291 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:23.103895 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" Jul 18 01:55:23.104291 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:23.103923 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv" event={"ID":"22680e42-703c-483d-8bf3-ab25b40c43c4","Type":"ContainerDied","Data":"baf6d595ff81c6e88035228a5b5f9db0a31dd18d6ef8b81bc0d757468fa5a905"} Jul 18 01:55:23.104291 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:23.103954 2570 scope.go:117] "RemoveContainer" containerID="6a028e3a0a788007394e95dd9575adb3f2533685eecb34fa4135f5b546e927b4" Jul 18 01:55:23.112576 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:23.112557 2570 scope.go:117] "RemoveContainer" containerID="6a028e3a0a788007394e95dd9575adb3f2533685eecb34fa4135f5b546e927b4" Jul 18 01:55:23.112857 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:55:23.112838 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a028e3a0a788007394e95dd9575adb3f2533685eecb34fa4135f5b546e927b4\": container with ID starting with 6a028e3a0a788007394e95dd9575adb3f2533685eecb34fa4135f5b546e927b4 not found: ID does not exist" containerID="6a028e3a0a788007394e95dd9575adb3f2533685eecb34fa4135f5b546e927b4" Jul 18 01:55:23.112915 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:23.112869 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a028e3a0a788007394e95dd9575adb3f2533685eecb34fa4135f5b546e927b4"} err="failed to get container status \"6a028e3a0a788007394e95dd9575adb3f2533685eecb34fa4135f5b546e927b4\": rpc error: code = NotFound desc = could not find container \"6a028e3a0a788007394e95dd9575adb3f2533685eecb34fa4135f5b546e927b4\": container with ID starting with 6a028e3a0a788007394e95dd9575adb3f2533685eecb34fa4135f5b546e927b4 not found: ID does not exist" Jul 18 01:55:23.121491 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:23.121457 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv"] Jul 18 01:55:23.129808 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:23.129781 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-f7lzv"] Jul 18 01:55:24.894586 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:24.894550 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22680e42-703c-483d-8bf3-ab25b40c43c4" path="/var/lib/kubelet/pods/22680e42-703c-483d-8bf3-ab25b40c43c4/volumes" Jul 18 01:55:26.489432 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.489402 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm"] Jul 18 01:55:26.489820 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.489784 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22680e42-703c-483d-8bf3-ab25b40c43c4" containerName="manager" Jul 18 01:55:26.489820 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.489796 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="22680e42-703c-483d-8bf3-ab25b40c43c4" containerName="manager" Jul 18 01:55:26.489820 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.489813 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a86969b-dcb5-4cf7-8007-dd9ca721b6df" containerName="console" Jul 18 01:55:26.489820 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.489821 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a86969b-dcb5-4cf7-8007-dd9ca721b6df" containerName="console" Jul 18 01:55:26.489947 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.489883 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a86969b-dcb5-4cf7-8007-dd9ca721b6df" containerName="console" Jul 18 01:55:26.489947 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.489898 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="22680e42-703c-483d-8bf3-ab25b40c43c4" containerName="manager" Jul 18 01:55:26.494059 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.494039 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.497286 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.497270 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-z6w9n\"" Jul 18 01:55:26.510460 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.510438 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm"] Jul 18 01:55:26.616011 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.615976 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be7b9a46-e566-4a97-a560-4d0e070934fe-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.616186 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.616014 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be7b9a46-e566-4a97-a560-4d0e070934fe-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.616186 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.616108 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzhrv\" (UniqueName: \"kubernetes.io/projected/be7b9a46-e566-4a97-a560-4d0e070934fe-kube-api-access-nzhrv\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.616186 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.616139 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be7b9a46-e566-4a97-a560-4d0e070934fe-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.616186 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.616163 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be7b9a46-e566-4a97-a560-4d0e070934fe-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.616333 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.616190 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be7b9a46-e566-4a97-a560-4d0e070934fe-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.616333 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.616247 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be7b9a46-e566-4a97-a560-4d0e070934fe-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.616333 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.616273 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7b9a46-e566-4a97-a560-4d0e070934fe-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.616333 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.616310 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7b9a46-e566-4a97-a560-4d0e070934fe-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.717453 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.717412 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be7b9a46-e566-4a97-a560-4d0e070934fe-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.717453 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.717451 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be7b9a46-e566-4a97-a560-4d0e070934fe-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.717729 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.717496 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzhrv\" (UniqueName: \"kubernetes.io/projected/be7b9a46-e566-4a97-a560-4d0e070934fe-kube-api-access-nzhrv\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.717729 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.717515 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be7b9a46-e566-4a97-a560-4d0e070934fe-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.717729 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.717537 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be7b9a46-e566-4a97-a560-4d0e070934fe-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.717729 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.717560 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be7b9a46-e566-4a97-a560-4d0e070934fe-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.717729 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.717592 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be7b9a46-e566-4a97-a560-4d0e070934fe-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.717729 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.717618 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7b9a46-e566-4a97-a560-4d0e070934fe-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.717729 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.717700 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7b9a46-e566-4a97-a560-4d0e070934fe-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.718129 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.718032 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be7b9a46-e566-4a97-a560-4d0e070934fe-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.718129 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.718071 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7b9a46-e566-4a97-a560-4d0e070934fe-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.718239 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.718158 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be7b9a46-e566-4a97-a560-4d0e070934fe-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.718366 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.718348 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be7b9a46-e566-4a97-a560-4d0e070934fe-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.718428 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.718378 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be7b9a46-e566-4a97-a560-4d0e070934fe-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.720060 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.720038 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be7b9a46-e566-4a97-a560-4d0e070934fe-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.720268 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.720251 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be7b9a46-e566-4a97-a560-4d0e070934fe-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.726605 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.726575 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzhrv\" (UniqueName: \"kubernetes.io/projected/be7b9a46-e566-4a97-a560-4d0e070934fe-kube-api-access-nzhrv\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.735260 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.735232 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be7b9a46-e566-4a97-a560-4d0e070934fe-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-gf7sm\" (UID: \"be7b9a46-e566-4a97-a560-4d0e070934fe\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.804434 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.804398 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:26.945109 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.945082 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm"] Jul 18 01:55:26.947292 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:55:26.947271 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe7b9a46_e566_4a97_a560_4d0e070934fe.slice/crio-851b9e6589194dff9302df7d38fb47a40dc4456f8f3f00f945fb8dbc5f595e97 WatchSource:0}: Error finding container 851b9e6589194dff9302df7d38fb47a40dc4456f8f3f00f945fb8dbc5f595e97: Status 404 returned error can't find the container with id 851b9e6589194dff9302df7d38fb47a40dc4456f8f3f00f945fb8dbc5f595e97 Jul 18 01:55:26.949321 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.949288 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:55:26.949413 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.949360 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:55:26.949413 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:26.949387 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:55:27.125284 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:27.125197 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" event={"ID":"be7b9a46-e566-4a97-a560-4d0e070934fe","Type":"ContainerStarted","Data":"c0e2076d83b122484b5273d51d13c3a092e40f6a11f6a3cbaacdb848ed4df16c"} Jul 18 01:55:27.125284 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:27.125259 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" event={"ID":"be7b9a46-e566-4a97-a560-4d0e070934fe","Type":"ContainerStarted","Data":"851b9e6589194dff9302df7d38fb47a40dc4456f8f3f00f945fb8dbc5f595e97"} Jul 18 01:55:27.151275 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:27.151221 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" podStartSLOduration=1.151204034 podStartE2EDuration="1.151204034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:27.150599436 +0000 UTC m=+466.828023547" watchObservedRunningTime="2026-07-18 01:55:27.151204034 +0000 UTC m=+466.828628145" Jul 18 01:55:27.804576 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:27.804537 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:28.809243 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:28.809209 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:29.135290 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:29.135201 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:29.136185 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:29.136167 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:55:40.130179 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:40.130141 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-v98f7"] Jul 18 01:55:40.133532 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:40.133514 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-v98f7" Jul 18 01:55:40.136059 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:40.136029 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bsnwq\"" Jul 18 01:55:40.141704 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:40.141661 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-v98f7"] Jul 18 01:55:40.228259 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:40.228221 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g66s\" (UniqueName: \"kubernetes.io/projected/f109aa76-3b77-472a-bba6-436dd15a95f7-kube-api-access-4g66s\") pod \"authorino-58cd955675-v98f7\" (UID: \"f109aa76-3b77-472a-bba6-436dd15a95f7\") " pod="kuadrant-system/authorino-58cd955675-v98f7" Jul 18 01:55:40.265444 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:40.265406 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-5869f"] Jul 18 01:55:40.268860 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:40.268836 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-5869f" Jul 18 01:55:40.274088 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:40.274066 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5869f"] Jul 18 01:55:40.329182 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:40.329148 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glhz7\" (UniqueName: \"kubernetes.io/projected/2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2-kube-api-access-glhz7\") pod \"authorino-75d556bd77-5869f\" (UID: \"2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2\") " pod="kuadrant-system/authorino-75d556bd77-5869f" Jul 18 01:55:40.329360 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:40.329220 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g66s\" (UniqueName: \"kubernetes.io/projected/f109aa76-3b77-472a-bba6-436dd15a95f7-kube-api-access-4g66s\") pod \"authorino-58cd955675-v98f7\" (UID: \"f109aa76-3b77-472a-bba6-436dd15a95f7\") " pod="kuadrant-system/authorino-58cd955675-v98f7" Jul 18 01:55:40.337814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:40.337778 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g66s\" (UniqueName: \"kubernetes.io/projected/f109aa76-3b77-472a-bba6-436dd15a95f7-kube-api-access-4g66s\") pod \"authorino-58cd955675-v98f7\" (UID: \"f109aa76-3b77-472a-bba6-436dd15a95f7\") " pod="kuadrant-system/authorino-58cd955675-v98f7" Jul 18 01:55:40.430710 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:40.430592 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glhz7\" (UniqueName: \"kubernetes.io/projected/2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2-kube-api-access-glhz7\") pod \"authorino-75d556bd77-5869f\" (UID: \"2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2\") " pod="kuadrant-system/authorino-75d556bd77-5869f" Jul 18 01:55:40.439168 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:40.439131 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glhz7\" (UniqueName: \"kubernetes.io/projected/2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2-kube-api-access-glhz7\") pod \"authorino-75d556bd77-5869f\" (UID: \"2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2\") " pod="kuadrant-system/authorino-75d556bd77-5869f" Jul 18 01:55:40.445929 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:40.445902 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-v98f7" Jul 18 01:55:40.578966 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:40.578924 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-5869f" Jul 18 01:55:40.585219 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:40.585188 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-v98f7"] Jul 18 01:55:40.589390 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:55:40.589356 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf109aa76_3b77_472a_bba6_436dd15a95f7.slice/crio-895dc2024556e59dcabe6fdcec7a09663d7b3805673f683ce7696ff325b4ba64 WatchSource:0}: Error finding container 895dc2024556e59dcabe6fdcec7a09663d7b3805673f683ce7696ff325b4ba64: Status 404 returned error can't find the container with id 895dc2024556e59dcabe6fdcec7a09663d7b3805673f683ce7696ff325b4ba64 Jul 18 01:55:40.717286 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:40.717258 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5869f"] Jul 18 01:55:40.719381 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:55:40.719352 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d9f778f_a5da_4ec5_9e2c_3f8ffd2033a2.slice/crio-e6d7efaf515dce8c104272b45105c75a8b8cd643df97fea32ad841efb9443af9 WatchSource:0}: Error finding container e6d7efaf515dce8c104272b45105c75a8b8cd643df97fea32ad841efb9443af9: Status 404 returned error can't find the container with id e6d7efaf515dce8c104272b45105c75a8b8cd643df97fea32ad841efb9443af9 Jul 18 01:55:41.201021 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:41.200980 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-v98f7" event={"ID":"f109aa76-3b77-472a-bba6-436dd15a95f7","Type":"ContainerStarted","Data":"895dc2024556e59dcabe6fdcec7a09663d7b3805673f683ce7696ff325b4ba64"} Jul 18 01:55:41.202632 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:41.202594 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-5869f" event={"ID":"2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2","Type":"ContainerStarted","Data":"e6d7efaf515dce8c104272b45105c75a8b8cd643df97fea32ad841efb9443af9"} Jul 18 01:55:44.225871 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:44.225827 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-v98f7" event={"ID":"f109aa76-3b77-472a-bba6-436dd15a95f7","Type":"ContainerStarted","Data":"05420effdee6073de5572b4d18c0e9d8f17b76bbfcce35e221424b555ee8cacd"} Jul 18 01:55:44.227732 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:44.227684 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-5869f" event={"ID":"2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2","Type":"ContainerStarted","Data":"ffc9eb7ef9e6b9786c57dd506813ba6b380e9c6c45a4fa3b02d31c788b97d6ea"} Jul 18 01:55:44.241520 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:44.241469 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-v98f7" podStartSLOduration=0.775833951 podStartE2EDuration="4.241454017s" totalImagesPullingTime="3.465620066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:44.240772297 +0000 UTC m=+483.918196408" watchObservedRunningTime="2026-07-18 01:55:44.241454017 +0000 UTC m=+483.918878127" Jul 18 01:55:44.266197 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:44.266132 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-5869f" podStartSLOduration=0.918289003 podStartE2EDuration="4.26611017s" totalImagesPullingTime="3.347821167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:44.264283415 +0000 UTC m=+483.941707528" watchObservedRunningTime="2026-07-18 01:55:44.26611017 +0000 UTC m=+483.943534282" Jul 18 01:55:44.335209 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:44.335120 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-v98f7"] Jul 18 01:55:46.236776 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:46.236733 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-v98f7" podUID="f109aa76-3b77-472a-bba6-436dd15a95f7" containerName="authorino" containerID="cri-o://05420effdee6073de5572b4d18c0e9d8f17b76bbfcce35e221424b555ee8cacd" gracePeriod=30 Jul 18 01:55:46.491562 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:46.491499 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-v98f7" Jul 18 01:55:46.589203 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:46.589174 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4g66s\" (UniqueName: \"kubernetes.io/projected/f109aa76-3b77-472a-bba6-436dd15a95f7-kube-api-access-4g66s\") pod \"f109aa76-3b77-472a-bba6-436dd15a95f7\" (UID: \"f109aa76-3b77-472a-bba6-436dd15a95f7\") " Jul 18 01:55:46.591227 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:46.591202 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f109aa76-3b77-472a-bba6-436dd15a95f7-kube-api-access-4g66s" (OuterVolumeSpecName: "kube-api-access-4g66s") pod "f109aa76-3b77-472a-bba6-436dd15a95f7" (UID: "f109aa76-3b77-472a-bba6-436dd15a95f7"). InnerVolumeSpecName "kube-api-access-4g66s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:55:46.690631 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:46.690587 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4g66s\" (UniqueName: \"kubernetes.io/projected/f109aa76-3b77-472a-bba6-436dd15a95f7-kube-api-access-4g66s\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:55:47.242374 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:47.242280 2570 generic.go:356] "Generic (PLEG): container finished" podID="f109aa76-3b77-472a-bba6-436dd15a95f7" containerID="05420effdee6073de5572b4d18c0e9d8f17b76bbfcce35e221424b555ee8cacd" exitCode=0 Jul 18 01:55:47.242374 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:47.242335 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-v98f7" Jul 18 01:55:47.242931 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:47.242367 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-v98f7" event={"ID":"f109aa76-3b77-472a-bba6-436dd15a95f7","Type":"ContainerDied","Data":"05420effdee6073de5572b4d18c0e9d8f17b76bbfcce35e221424b555ee8cacd"} Jul 18 01:55:47.242931 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:47.242415 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-v98f7" event={"ID":"f109aa76-3b77-472a-bba6-436dd15a95f7","Type":"ContainerDied","Data":"895dc2024556e59dcabe6fdcec7a09663d7b3805673f683ce7696ff325b4ba64"} Jul 18 01:55:47.242931 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:47.242438 2570 scope.go:117] "RemoveContainer" containerID="05420effdee6073de5572b4d18c0e9d8f17b76bbfcce35e221424b555ee8cacd" Jul 18 01:55:47.250747 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:47.250732 2570 scope.go:117] "RemoveContainer" containerID="05420effdee6073de5572b4d18c0e9d8f17b76bbfcce35e221424b555ee8cacd" Jul 18 01:55:47.250986 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:55:47.250969 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05420effdee6073de5572b4d18c0e9d8f17b76bbfcce35e221424b555ee8cacd\": container with ID starting with 05420effdee6073de5572b4d18c0e9d8f17b76bbfcce35e221424b555ee8cacd not found: ID does not exist" containerID="05420effdee6073de5572b4d18c0e9d8f17b76bbfcce35e221424b555ee8cacd" Jul 18 01:55:47.251056 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:47.250993 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05420effdee6073de5572b4d18c0e9d8f17b76bbfcce35e221424b555ee8cacd"} err="failed to get container status \"05420effdee6073de5572b4d18c0e9d8f17b76bbfcce35e221424b555ee8cacd\": rpc error: code = NotFound desc = could not find container \"05420effdee6073de5572b4d18c0e9d8f17b76bbfcce35e221424b555ee8cacd\": container with ID starting with 05420effdee6073de5572b4d18c0e9d8f17b76bbfcce35e221424b555ee8cacd not found: ID does not exist" Jul 18 01:55:47.260054 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:47.260029 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-v98f7"] Jul 18 01:55:47.267248 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:47.267220 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-v98f7"] Jul 18 01:55:48.894322 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:55:48.894286 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f109aa76-3b77-472a-bba6-436dd15a95f7" path="/var/lib/kubelet/pods/f109aa76-3b77-472a-bba6-436dd15a95f7/volumes" Jul 18 01:56:12.404159 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.404123 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jz77s"] Jul 18 01:56:12.404598 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.404527 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f109aa76-3b77-472a-bba6-436dd15a95f7" containerName="authorino" Jul 18 01:56:12.404598 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.404539 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f109aa76-3b77-472a-bba6-436dd15a95f7" containerName="authorino" Jul 18 01:56:12.404710 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.404604 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f109aa76-3b77-472a-bba6-436dd15a95f7" containerName="authorino" Jul 18 01:56:12.408811 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.408783 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jz77s" Jul 18 01:56:12.421081 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.421051 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jz77s"] Jul 18 01:56:12.521499 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.521469 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvlmd\" (UniqueName: \"kubernetes.io/projected/8747357f-5a76-4a61-bfe2-43a6a8ee88a7-kube-api-access-zvlmd\") pod \"authorino-5d4b67c446-jz77s\" (UID: \"8747357f-5a76-4a61-bfe2-43a6a8ee88a7\") " pod="kuadrant-system/authorino-5d4b67c446-jz77s" Jul 18 01:56:12.622944 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.622890 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvlmd\" (UniqueName: \"kubernetes.io/projected/8747357f-5a76-4a61-bfe2-43a6a8ee88a7-kube-api-access-zvlmd\") pod \"authorino-5d4b67c446-jz77s\" (UID: \"8747357f-5a76-4a61-bfe2-43a6a8ee88a7\") " pod="kuadrant-system/authorino-5d4b67c446-jz77s" Jul 18 01:56:12.640757 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.640720 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvlmd\" (UniqueName: \"kubernetes.io/projected/8747357f-5a76-4a61-bfe2-43a6a8ee88a7-kube-api-access-zvlmd\") pod \"authorino-5d4b67c446-jz77s\" (UID: \"8747357f-5a76-4a61-bfe2-43a6a8ee88a7\") " pod="kuadrant-system/authorino-5d4b67c446-jz77s" Jul 18 01:56:12.679362 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.679275 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jz77s"] Jul 18 01:56:12.679612 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.679594 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jz77s" Jul 18 01:56:12.702347 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.702315 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-77957b5686-phnd4"] Jul 18 01:56:12.705578 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.705555 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77957b5686-phnd4" Jul 18 01:56:12.715501 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.715476 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-77957b5686-phnd4"] Jul 18 01:56:12.817303 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.817278 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jz77s"] Jul 18 01:56:12.819151 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:56:12.819125 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8747357f_5a76_4a61_bfe2_43a6a8ee88a7.slice/crio-070ddc445f505a9738c4ef3c79a79c5e216ae0efff0c54ec12b72559e1488248 WatchSource:0}: Error finding container 070ddc445f505a9738c4ef3c79a79c5e216ae0efff0c54ec12b72559e1488248: Status 404 returned error can't find the container with id 070ddc445f505a9738c4ef3c79a79c5e216ae0efff0c54ec12b72559e1488248 Jul 18 01:56:12.824447 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.824426 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g4mn\" (UniqueName: \"kubernetes.io/projected/049be3d4-f15b-4a3c-9396-965dcb4577a0-kube-api-access-5g4mn\") pod \"authorino-77957b5686-phnd4\" (UID: \"049be3d4-f15b-4a3c-9396-965dcb4577a0\") " pod="kuadrant-system/authorino-77957b5686-phnd4" Jul 18 01:56:12.861714 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.861680 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-77957b5686-phnd4"] Jul 18 01:56:12.861945 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:56:12.861926 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-5g4mn], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-77957b5686-phnd4" podUID="049be3d4-f15b-4a3c-9396-965dcb4577a0" Jul 18 01:56:12.925856 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.925816 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g4mn\" (UniqueName: \"kubernetes.io/projected/049be3d4-f15b-4a3c-9396-965dcb4577a0-kube-api-access-5g4mn\") pod \"authorino-77957b5686-phnd4\" (UID: \"049be3d4-f15b-4a3c-9396-965dcb4577a0\") " pod="kuadrant-system/authorino-77957b5686-phnd4" Jul 18 01:56:12.948222 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:12.948149 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g4mn\" (UniqueName: \"kubernetes.io/projected/049be3d4-f15b-4a3c-9396-965dcb4577a0-kube-api-access-5g4mn\") pod \"authorino-77957b5686-phnd4\" (UID: \"049be3d4-f15b-4a3c-9396-965dcb4577a0\") " pod="kuadrant-system/authorino-77957b5686-phnd4" Jul 18 01:56:13.368292 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:13.368243 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jz77s" event={"ID":"8747357f-5a76-4a61-bfe2-43a6a8ee88a7","Type":"ContainerStarted","Data":"8a5ceeae8e1b5eaa0c911a4e06f6d3c87278fbd85bd70e3f258dde636a767c28"} Jul 18 01:56:13.368292 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:13.368265 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-jz77s" podUID="8747357f-5a76-4a61-bfe2-43a6a8ee88a7" containerName="authorino" containerID="cri-o://8a5ceeae8e1b5eaa0c911a4e06f6d3c87278fbd85bd70e3f258dde636a767c28" gracePeriod=30 Jul 18 01:56:13.368574 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:13.368303 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jz77s" event={"ID":"8747357f-5a76-4a61-bfe2-43a6a8ee88a7","Type":"ContainerStarted","Data":"070ddc445f505a9738c4ef3c79a79c5e216ae0efff0c54ec12b72559e1488248"} Jul 18 01:56:13.368574 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:13.368266 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77957b5686-phnd4" Jul 18 01:56:13.375837 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:13.375814 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77957b5686-phnd4" Jul 18 01:56:13.384064 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:13.384020 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-jz77s" podStartSLOduration=0.982668881 podStartE2EDuration="1.38399747s" totalImagesPullingTime="401.328589ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:13.383052263 +0000 UTC m=+513.060476373" watchObservedRunningTime="2026-07-18 01:56:13.38399747 +0000 UTC m=+513.061421621" Jul 18 01:56:13.531272 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:13.531230 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5g4mn\" (UniqueName: \"kubernetes.io/projected/049be3d4-f15b-4a3c-9396-965dcb4577a0-kube-api-access-5g4mn\") pod \"049be3d4-f15b-4a3c-9396-965dcb4577a0\" (UID: \"049be3d4-f15b-4a3c-9396-965dcb4577a0\") " Jul 18 01:56:13.533456 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:13.533425 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/049be3d4-f15b-4a3c-9396-965dcb4577a0-kube-api-access-5g4mn" (OuterVolumeSpecName: "kube-api-access-5g4mn") pod "049be3d4-f15b-4a3c-9396-965dcb4577a0" (UID: "049be3d4-f15b-4a3c-9396-965dcb4577a0"). InnerVolumeSpecName "kube-api-access-5g4mn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:56:13.609395 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:13.609370 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jz77s" Jul 18 01:56:13.632370 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:13.632328 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g4mn\" (UniqueName: \"kubernetes.io/projected/049be3d4-f15b-4a3c-9396-965dcb4577a0-kube-api-access-5g4mn\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:56:13.732758 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:13.732633 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvlmd\" (UniqueName: \"kubernetes.io/projected/8747357f-5a76-4a61-bfe2-43a6a8ee88a7-kube-api-access-zvlmd\") pod \"8747357f-5a76-4a61-bfe2-43a6a8ee88a7\" (UID: \"8747357f-5a76-4a61-bfe2-43a6a8ee88a7\") " Jul 18 01:56:13.734902 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:13.734863 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8747357f-5a76-4a61-bfe2-43a6a8ee88a7-kube-api-access-zvlmd" (OuterVolumeSpecName: "kube-api-access-zvlmd") pod "8747357f-5a76-4a61-bfe2-43a6a8ee88a7" (UID: "8747357f-5a76-4a61-bfe2-43a6a8ee88a7"). InnerVolumeSpecName "kube-api-access-zvlmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:56:13.833962 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:13.833921 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvlmd\" (UniqueName: \"kubernetes.io/projected/8747357f-5a76-4a61-bfe2-43a6a8ee88a7-kube-api-access-zvlmd\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:56:14.374589 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:14.374551 2570 generic.go:356] "Generic (PLEG): container finished" podID="8747357f-5a76-4a61-bfe2-43a6a8ee88a7" containerID="8a5ceeae8e1b5eaa0c911a4e06f6d3c87278fbd85bd70e3f258dde636a767c28" exitCode=0 Jul 18 01:56:14.374805 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:14.374599 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jz77s" Jul 18 01:56:14.374805 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:14.374665 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jz77s" event={"ID":"8747357f-5a76-4a61-bfe2-43a6a8ee88a7","Type":"ContainerDied","Data":"8a5ceeae8e1b5eaa0c911a4e06f6d3c87278fbd85bd70e3f258dde636a767c28"} Jul 18 01:56:14.374805 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:14.374710 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jz77s" event={"ID":"8747357f-5a76-4a61-bfe2-43a6a8ee88a7","Type":"ContainerDied","Data":"070ddc445f505a9738c4ef3c79a79c5e216ae0efff0c54ec12b72559e1488248"} Jul 18 01:56:14.374805 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:14.374728 2570 scope.go:117] "RemoveContainer" containerID="8a5ceeae8e1b5eaa0c911a4e06f6d3c87278fbd85bd70e3f258dde636a767c28" Jul 18 01:56:14.375011 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:14.374883 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-77957b5686-phnd4" Jul 18 01:56:14.384626 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:14.384609 2570 scope.go:117] "RemoveContainer" containerID="8a5ceeae8e1b5eaa0c911a4e06f6d3c87278fbd85bd70e3f258dde636a767c28" Jul 18 01:56:14.384960 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:56:14.384922 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a5ceeae8e1b5eaa0c911a4e06f6d3c87278fbd85bd70e3f258dde636a767c28\": container with ID starting with 8a5ceeae8e1b5eaa0c911a4e06f6d3c87278fbd85bd70e3f258dde636a767c28 not found: ID does not exist" containerID="8a5ceeae8e1b5eaa0c911a4e06f6d3c87278fbd85bd70e3f258dde636a767c28" Jul 18 01:56:14.384999 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:14.384972 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a5ceeae8e1b5eaa0c911a4e06f6d3c87278fbd85bd70e3f258dde636a767c28"} err="failed to get container status \"8a5ceeae8e1b5eaa0c911a4e06f6d3c87278fbd85bd70e3f258dde636a767c28\": rpc error: code = NotFound desc = could not find container \"8a5ceeae8e1b5eaa0c911a4e06f6d3c87278fbd85bd70e3f258dde636a767c28\": container with ID starting with 8a5ceeae8e1b5eaa0c911a4e06f6d3c87278fbd85bd70e3f258dde636a767c28 not found: ID does not exist" Jul 18 01:56:14.416057 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:14.416027 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-77957b5686-phnd4"] Jul 18 01:56:14.436820 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:14.436789 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-77957b5686-phnd4"] Jul 18 01:56:14.447757 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:14.447730 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jz77s"] Jul 18 01:56:14.450427 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:14.450404 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jz77s"] Jul 18 01:56:14.894496 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:14.894460 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="049be3d4-f15b-4a3c-9396-965dcb4577a0" path="/var/lib/kubelet/pods/049be3d4-f15b-4a3c-9396-965dcb4577a0/volumes" Jul 18 01:56:14.894904 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:14.894709 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8747357f-5a76-4a61-bfe2-43a6a8ee88a7" path="/var/lib/kubelet/pods/8747357f-5a76-4a61-bfe2-43a6a8ee88a7/volumes" Jul 18 01:56:16.921033 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:16.920991 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5869f"] Jul 18 01:56:16.921564 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:16.921257 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-5869f" podUID="2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2" containerName="authorino" containerID="cri-o://ffc9eb7ef9e6b9786c57dd506813ba6b380e9c6c45a4fa3b02d31c788b97d6ea" gracePeriod=30 Jul 18 01:56:17.197553 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:17.197527 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-5869f" Jul 18 01:56:17.367342 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:17.367307 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glhz7\" (UniqueName: \"kubernetes.io/projected/2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2-kube-api-access-glhz7\") pod \"2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2\" (UID: \"2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2\") " Jul 18 01:56:17.369406 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:17.369376 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2-kube-api-access-glhz7" (OuterVolumeSpecName: "kube-api-access-glhz7") pod "2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2" (UID: "2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2"). InnerVolumeSpecName "kube-api-access-glhz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:56:17.390585 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:17.390547 2570 generic.go:356] "Generic (PLEG): container finished" podID="2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2" containerID="ffc9eb7ef9e6b9786c57dd506813ba6b380e9c6c45a4fa3b02d31c788b97d6ea" exitCode=0 Jul 18 01:56:17.390744 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:17.390590 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-5869f" Jul 18 01:56:17.390744 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:17.390588 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-5869f" event={"ID":"2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2","Type":"ContainerDied","Data":"ffc9eb7ef9e6b9786c57dd506813ba6b380e9c6c45a4fa3b02d31c788b97d6ea"} Jul 18 01:56:17.390744 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:17.390699 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-5869f" event={"ID":"2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2","Type":"ContainerDied","Data":"e6d7efaf515dce8c104272b45105c75a8b8cd643df97fea32ad841efb9443af9"} Jul 18 01:56:17.390744 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:17.390716 2570 scope.go:117] "RemoveContainer" containerID="ffc9eb7ef9e6b9786c57dd506813ba6b380e9c6c45a4fa3b02d31c788b97d6ea" Jul 18 01:56:17.399484 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:17.399465 2570 scope.go:117] "RemoveContainer" containerID="ffc9eb7ef9e6b9786c57dd506813ba6b380e9c6c45a4fa3b02d31c788b97d6ea" Jul 18 01:56:17.399774 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:56:17.399754 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffc9eb7ef9e6b9786c57dd506813ba6b380e9c6c45a4fa3b02d31c788b97d6ea\": container with ID starting with ffc9eb7ef9e6b9786c57dd506813ba6b380e9c6c45a4fa3b02d31c788b97d6ea not found: ID does not exist" containerID="ffc9eb7ef9e6b9786c57dd506813ba6b380e9c6c45a4fa3b02d31c788b97d6ea" Jul 18 01:56:17.399826 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:17.399783 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffc9eb7ef9e6b9786c57dd506813ba6b380e9c6c45a4fa3b02d31c788b97d6ea"} err="failed to get container status \"ffc9eb7ef9e6b9786c57dd506813ba6b380e9c6c45a4fa3b02d31c788b97d6ea\": rpc error: code = NotFound desc = could not find container \"ffc9eb7ef9e6b9786c57dd506813ba6b380e9c6c45a4fa3b02d31c788b97d6ea\": container with ID starting with ffc9eb7ef9e6b9786c57dd506813ba6b380e9c6c45a4fa3b02d31c788b97d6ea not found: ID does not exist" Jul 18 01:56:17.411816 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:17.411790 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5869f"] Jul 18 01:56:17.415498 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:17.415470 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-5869f"] Jul 18 01:56:17.468331 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:17.468247 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glhz7\" (UniqueName: \"kubernetes.io/projected/2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2-kube-api-access-glhz7\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:56:18.895014 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:18.894979 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2" path="/var/lib/kubelet/pods/2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2/volumes" Jul 18 01:56:21.721427 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:21.721389 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-85bbc86896-pv72n"] Jul 18 01:56:21.721873 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:21.721796 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8747357f-5a76-4a61-bfe2-43a6a8ee88a7" containerName="authorino" Jul 18 01:56:21.721873 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:21.721809 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8747357f-5a76-4a61-bfe2-43a6a8ee88a7" containerName="authorino" Jul 18 01:56:21.721873 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:21.721824 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2" containerName="authorino" Jul 18 01:56:21.721873 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:21.721830 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2" containerName="authorino" Jul 18 01:56:21.721998 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:21.721887 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8747357f-5a76-4a61-bfe2-43a6a8ee88a7" containerName="authorino" Jul 18 01:56:21.721998 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:21.721894 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d9f778f-a5da-4ec5-9e2c-3f8ffd2033a2" containerName="authorino" Jul 18 01:56:21.726153 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:21.726131 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85bbc86896-pv72n" Jul 18 01:56:21.728756 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:21.728729 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 18 01:56:21.728896 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:21.728813 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nzhxz\"" Jul 18 01:56:21.728896 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:21.728813 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 18 01:56:21.741242 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:21.741210 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85bbc86896-pv72n"] Jul 18 01:56:21.879265 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:21.879233 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74bf56b8d6-ltm5j"] Jul 18 01:56:21.882752 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:21.882728 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" Jul 18 01:56:21.892195 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:21.892163 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74bf56b8d6-ltm5j"] Jul 18 01:56:21.908221 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:21.908186 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bda4d59a-9dd1-4fdf-8abd-22a672d53af1-webhook-cert\") pod \"maas-controller-85bbc86896-pv72n\" (UID: \"bda4d59a-9dd1-4fdf-8abd-22a672d53af1\") " pod="opendatahub/maas-controller-85bbc86896-pv72n" Jul 18 01:56:21.908406 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:21.908241 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrl4x\" (UniqueName: \"kubernetes.io/projected/bda4d59a-9dd1-4fdf-8abd-22a672d53af1-kube-api-access-mrl4x\") pod \"maas-controller-85bbc86896-pv72n\" (UID: \"bda4d59a-9dd1-4fdf-8abd-22a672d53af1\") " pod="opendatahub/maas-controller-85bbc86896-pv72n" Jul 18 01:56:22.009033 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:22.008928 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrl4x\" (UniqueName: \"kubernetes.io/projected/bda4d59a-9dd1-4fdf-8abd-22a672d53af1-kube-api-access-mrl4x\") pod \"maas-controller-85bbc86896-pv72n\" (UID: \"bda4d59a-9dd1-4fdf-8abd-22a672d53af1\") " pod="opendatahub/maas-controller-85bbc86896-pv72n" Jul 18 01:56:22.009033 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:22.008981 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dczr\" (UniqueName: \"kubernetes.io/projected/86161998-684e-4b4d-b23c-015a7581158d-kube-api-access-6dczr\") pod \"maas-controller-74bf56b8d6-ltm5j\" (UID: \"86161998-684e-4b4d-b23c-015a7581158d\") " pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" Jul 18 01:56:22.009033 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:22.009027 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86161998-684e-4b4d-b23c-015a7581158d-webhook-cert\") pod \"maas-controller-74bf56b8d6-ltm5j\" (UID: \"86161998-684e-4b4d-b23c-015a7581158d\") " pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" Jul 18 01:56:22.009316 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:22.009130 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bda4d59a-9dd1-4fdf-8abd-22a672d53af1-webhook-cert\") pod \"maas-controller-85bbc86896-pv72n\" (UID: \"bda4d59a-9dd1-4fdf-8abd-22a672d53af1\") " pod="opendatahub/maas-controller-85bbc86896-pv72n" Jul 18 01:56:22.011450 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:22.011430 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bda4d59a-9dd1-4fdf-8abd-22a672d53af1-webhook-cert\") pod \"maas-controller-85bbc86896-pv72n\" (UID: \"bda4d59a-9dd1-4fdf-8abd-22a672d53af1\") " pod="opendatahub/maas-controller-85bbc86896-pv72n" Jul 18 01:56:22.025527 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:22.025503 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrl4x\" (UniqueName: \"kubernetes.io/projected/bda4d59a-9dd1-4fdf-8abd-22a672d53af1-kube-api-access-mrl4x\") pod \"maas-controller-85bbc86896-pv72n\" (UID: \"bda4d59a-9dd1-4fdf-8abd-22a672d53af1\") " pod="opendatahub/maas-controller-85bbc86896-pv72n" Jul 18 01:56:22.036404 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:22.036367 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85bbc86896-pv72n" Jul 18 01:56:22.110600 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:22.110571 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dczr\" (UniqueName: \"kubernetes.io/projected/86161998-684e-4b4d-b23c-015a7581158d-kube-api-access-6dczr\") pod \"maas-controller-74bf56b8d6-ltm5j\" (UID: \"86161998-684e-4b4d-b23c-015a7581158d\") " pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" Jul 18 01:56:22.110776 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:22.110625 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86161998-684e-4b4d-b23c-015a7581158d-webhook-cert\") pod \"maas-controller-74bf56b8d6-ltm5j\" (UID: \"86161998-684e-4b4d-b23c-015a7581158d\") " pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" Jul 18 01:56:22.113467 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:22.113382 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86161998-684e-4b4d-b23c-015a7581158d-webhook-cert\") pod \"maas-controller-74bf56b8d6-ltm5j\" (UID: \"86161998-684e-4b4d-b23c-015a7581158d\") " pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" Jul 18 01:56:22.131033 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:22.131001 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dczr\" (UniqueName: \"kubernetes.io/projected/86161998-684e-4b4d-b23c-015a7581158d-kube-api-access-6dczr\") pod \"maas-controller-74bf56b8d6-ltm5j\" (UID: \"86161998-684e-4b4d-b23c-015a7581158d\") " pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" Jul 18 01:56:22.179284 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:22.179259 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85bbc86896-pv72n"] Jul 18 01:56:22.181247 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:56:22.181217 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda4d59a_9dd1_4fdf_8abd_22a672d53af1.slice/crio-91fcd37212955f5fdf8ca1a64e20fa280146148e38940e89f12b9f26e9d0b36b WatchSource:0}: Error finding container 91fcd37212955f5fdf8ca1a64e20fa280146148e38940e89f12b9f26e9d0b36b: Status 404 returned error can't find the container with id 91fcd37212955f5fdf8ca1a64e20fa280146148e38940e89f12b9f26e9d0b36b Jul 18 01:56:22.193501 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:22.193481 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" Jul 18 01:56:22.328001 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:22.327977 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74bf56b8d6-ltm5j"] Jul 18 01:56:22.329963 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:56:22.329940 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86161998_684e_4b4d_b23c_015a7581158d.slice/crio-49f69fe85bb13faf26e2fab2ee8c8ad33aba3808cbef8d895fff9e643ab4ad91 WatchSource:0}: Error finding container 49f69fe85bb13faf26e2fab2ee8c8ad33aba3808cbef8d895fff9e643ab4ad91: Status 404 returned error can't find the container with id 49f69fe85bb13faf26e2fab2ee8c8ad33aba3808cbef8d895fff9e643ab4ad91 Jul 18 01:56:22.416463 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:22.416428 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" event={"ID":"86161998-684e-4b4d-b23c-015a7581158d","Type":"ContainerStarted","Data":"49f69fe85bb13faf26e2fab2ee8c8ad33aba3808cbef8d895fff9e643ab4ad91"} Jul 18 01:56:22.417498 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:22.417471 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85bbc86896-pv72n" event={"ID":"bda4d59a-9dd1-4fdf-8abd-22a672d53af1","Type":"ContainerStarted","Data":"91fcd37212955f5fdf8ca1a64e20fa280146148e38940e89f12b9f26e9d0b36b"} Jul 18 01:56:25.437246 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:25.437205 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" event={"ID":"86161998-684e-4b4d-b23c-015a7581158d","Type":"ContainerStarted","Data":"6c87042c8880b9d3877afeb8aadf8a2e3daf41b665c072c500e53f60b0cc9ddc"} Jul 18 01:56:25.437692 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:25.437314 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" Jul 18 01:56:25.438801 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:25.438777 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85bbc86896-pv72n" event={"ID":"bda4d59a-9dd1-4fdf-8abd-22a672d53af1","Type":"ContainerStarted","Data":"e2ac7a702bdb8594e927f267f0872c4cb66f242ac0f85b9db5c5143cbb95aa8e"} Jul 18 01:56:25.438922 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:25.438826 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-85bbc86896-pv72n" Jul 18 01:56:25.462734 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:25.462683 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" podStartSLOduration=1.835367867 podStartE2EDuration="4.462667576s" totalImagesPullingTime="2.627299709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:25.460756678 +0000 UTC m=+525.138180790" watchObservedRunningTime="2026-07-18 01:56:25.462667576 +0000 UTC m=+525.140091712" Jul 18 01:56:25.486587 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:25.486532 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-85bbc86896-pv72n" podStartSLOduration=1.6997775979999998 podStartE2EDuration="4.486518411s" totalImagesPullingTime="2.786740813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:25.485051285 +0000 UTC m=+525.162475396" watchObservedRunningTime="2026-07-18 01:56:25.486518411 +0000 UTC m=+525.163942522" Jul 18 01:56:36.445616 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:36.445535 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-85bbc86896-pv72n" Jul 18 01:56:36.445616 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:36.445609 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" Jul 18 01:56:36.509868 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:36.509833 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85bbc86896-pv72n"] Jul 18 01:56:36.510081 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:36.510055 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-85bbc86896-pv72n" podUID="bda4d59a-9dd1-4fdf-8abd-22a672d53af1" containerName="manager" containerID="cri-o://e2ac7a702bdb8594e927f267f0872c4cb66f242ac0f85b9db5c5143cbb95aa8e" gracePeriod=30 Jul 18 01:56:36.758536 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:36.758511 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85bbc86896-pv72n" Jul 18 01:56:36.825424 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:36.825384 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrl4x\" (UniqueName: \"kubernetes.io/projected/bda4d59a-9dd1-4fdf-8abd-22a672d53af1-kube-api-access-mrl4x\") pod \"bda4d59a-9dd1-4fdf-8abd-22a672d53af1\" (UID: \"bda4d59a-9dd1-4fdf-8abd-22a672d53af1\") " Jul 18 01:56:36.825591 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:36.825446 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bda4d59a-9dd1-4fdf-8abd-22a672d53af1-webhook-cert\") pod \"bda4d59a-9dd1-4fdf-8abd-22a672d53af1\" (UID: \"bda4d59a-9dd1-4fdf-8abd-22a672d53af1\") " Jul 18 01:56:36.827676 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:36.827623 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bda4d59a-9dd1-4fdf-8abd-22a672d53af1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "bda4d59a-9dd1-4fdf-8abd-22a672d53af1" (UID: "bda4d59a-9dd1-4fdf-8abd-22a672d53af1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:56:36.827676 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:36.827668 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bda4d59a-9dd1-4fdf-8abd-22a672d53af1-kube-api-access-mrl4x" (OuterVolumeSpecName: "kube-api-access-mrl4x") pod "bda4d59a-9dd1-4fdf-8abd-22a672d53af1" (UID: "bda4d59a-9dd1-4fdf-8abd-22a672d53af1"). InnerVolumeSpecName "kube-api-access-mrl4x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:56:36.926545 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:36.926518 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrl4x\" (UniqueName: \"kubernetes.io/projected/bda4d59a-9dd1-4fdf-8abd-22a672d53af1-kube-api-access-mrl4x\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:56:36.926545 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:36.926544 2570 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bda4d59a-9dd1-4fdf-8abd-22a672d53af1-webhook-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 01:56:37.496814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:37.496780 2570 generic.go:356] "Generic (PLEG): container finished" podID="bda4d59a-9dd1-4fdf-8abd-22a672d53af1" containerID="e2ac7a702bdb8594e927f267f0872c4cb66f242ac0f85b9db5c5143cbb95aa8e" exitCode=0 Jul 18 01:56:37.497214 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:37.496821 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85bbc86896-pv72n" Jul 18 01:56:37.497214 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:37.496868 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85bbc86896-pv72n" event={"ID":"bda4d59a-9dd1-4fdf-8abd-22a672d53af1","Type":"ContainerDied","Data":"e2ac7a702bdb8594e927f267f0872c4cb66f242ac0f85b9db5c5143cbb95aa8e"} Jul 18 01:56:37.497214 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:37.496915 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85bbc86896-pv72n" event={"ID":"bda4d59a-9dd1-4fdf-8abd-22a672d53af1","Type":"ContainerDied","Data":"91fcd37212955f5fdf8ca1a64e20fa280146148e38940e89f12b9f26e9d0b36b"} Jul 18 01:56:37.497214 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:37.496938 2570 scope.go:117] "RemoveContainer" containerID="e2ac7a702bdb8594e927f267f0872c4cb66f242ac0f85b9db5c5143cbb95aa8e" Jul 18 01:56:37.505331 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:37.505313 2570 scope.go:117] "RemoveContainer" containerID="e2ac7a702bdb8594e927f267f0872c4cb66f242ac0f85b9db5c5143cbb95aa8e" Jul 18 01:56:37.505599 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:56:37.505579 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2ac7a702bdb8594e927f267f0872c4cb66f242ac0f85b9db5c5143cbb95aa8e\": container with ID starting with e2ac7a702bdb8594e927f267f0872c4cb66f242ac0f85b9db5c5143cbb95aa8e not found: ID does not exist" containerID="e2ac7a702bdb8594e927f267f0872c4cb66f242ac0f85b9db5c5143cbb95aa8e" Jul 18 01:56:37.505685 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:37.505609 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2ac7a702bdb8594e927f267f0872c4cb66f242ac0f85b9db5c5143cbb95aa8e"} err="failed to get container status \"e2ac7a702bdb8594e927f267f0872c4cb66f242ac0f85b9db5c5143cbb95aa8e\": rpc error: code = NotFound desc = could not find container \"e2ac7a702bdb8594e927f267f0872c4cb66f242ac0f85b9db5c5143cbb95aa8e\": container with ID starting with e2ac7a702bdb8594e927f267f0872c4cb66f242ac0f85b9db5c5143cbb95aa8e not found: ID does not exist" Jul 18 01:56:37.515887 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:37.515864 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85bbc86896-pv72n"] Jul 18 01:56:37.517817 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:37.517796 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-85bbc86896-pv72n"] Jul 18 01:56:38.894049 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:38.894014 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bda4d59a-9dd1-4fdf-8abd-22a672d53af1" path="/var/lib/kubelet/pods/bda4d59a-9dd1-4fdf-8abd-22a672d53af1/volumes" Jul 18 01:56:55.267372 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.267339 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh"] Jul 18 01:56:55.267868 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.267746 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bda4d59a-9dd1-4fdf-8abd-22a672d53af1" containerName="manager" Jul 18 01:56:55.267868 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.267759 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="bda4d59a-9dd1-4fdf-8abd-22a672d53af1" containerName="manager" Jul 18 01:56:55.267868 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.267829 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="bda4d59a-9dd1-4fdf-8abd-22a672d53af1" containerName="manager" Jul 18 01:56:55.276260 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.276234 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh" Jul 18 01:56:55.279045 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.279015 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 01:56:55.279251 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.279236 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 18 01:56:55.279826 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.279808 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 18 01:56:55.279908 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.279843 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sfxcd\"" Jul 18 01:56:55.292708 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.292684 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh"] Jul 18 01:56:55.386471 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.386424 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/807b01d3-a35e-40df-bbdc-7044cab2d0aa-maas-api-tls\") pod \"maas-api-5fc78b4c74-sbwsh\" (UID: \"807b01d3-a35e-40df-bbdc-7044cab2d0aa\") " pod="odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh" Jul 18 01:56:55.386686 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.386500 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/807b01d3-a35e-40df-bbdc-7044cab2d0aa-openshift-service-ca\") pod \"maas-api-5fc78b4c74-sbwsh\" (UID: \"807b01d3-a35e-40df-bbdc-7044cab2d0aa\") " pod="odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh" Jul 18 01:56:55.386686 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.386586 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgp9n\" (UniqueName: \"kubernetes.io/projected/807b01d3-a35e-40df-bbdc-7044cab2d0aa-kube-api-access-qgp9n\") pod \"maas-api-5fc78b4c74-sbwsh\" (UID: \"807b01d3-a35e-40df-bbdc-7044cab2d0aa\") " pod="odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh" Jul 18 01:56:55.487747 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.487712 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgp9n\" (UniqueName: \"kubernetes.io/projected/807b01d3-a35e-40df-bbdc-7044cab2d0aa-kube-api-access-qgp9n\") pod \"maas-api-5fc78b4c74-sbwsh\" (UID: \"807b01d3-a35e-40df-bbdc-7044cab2d0aa\") " pod="odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh" Jul 18 01:56:55.487928 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.487783 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/807b01d3-a35e-40df-bbdc-7044cab2d0aa-maas-api-tls\") pod \"maas-api-5fc78b4c74-sbwsh\" (UID: \"807b01d3-a35e-40df-bbdc-7044cab2d0aa\") " pod="odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh" Jul 18 01:56:55.487928 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.487819 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/807b01d3-a35e-40df-bbdc-7044cab2d0aa-openshift-service-ca\") pod \"maas-api-5fc78b4c74-sbwsh\" (UID: \"807b01d3-a35e-40df-bbdc-7044cab2d0aa\") " pod="odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh" Jul 18 01:56:55.488036 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:56:55.487952 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 18 01:56:55.488036 ip-10-0-132-151 kubenswrapper[2570]: E0718 01:56:55.488032 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/807b01d3-a35e-40df-bbdc-7044cab2d0aa-maas-api-tls podName:807b01d3-a35e-40df-bbdc-7044cab2d0aa nodeName:}" failed. No retries permitted until 2026-07-18 01:56:55.988010878 +0000 UTC m=+555.665434973 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/807b01d3-a35e-40df-bbdc-7044cab2d0aa-maas-api-tls") pod "maas-api-5fc78b4c74-sbwsh" (UID: "807b01d3-a35e-40df-bbdc-7044cab2d0aa") : secret "maas-api-serving-cert" not found Jul 18 01:56:55.488482 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.488458 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/807b01d3-a35e-40df-bbdc-7044cab2d0aa-openshift-service-ca\") pod \"maas-api-5fc78b4c74-sbwsh\" (UID: \"807b01d3-a35e-40df-bbdc-7044cab2d0aa\") " pod="odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh" Jul 18 01:56:55.496372 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.496346 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgp9n\" (UniqueName: \"kubernetes.io/projected/807b01d3-a35e-40df-bbdc-7044cab2d0aa-kube-api-access-qgp9n\") pod \"maas-api-5fc78b4c74-sbwsh\" (UID: \"807b01d3-a35e-40df-bbdc-7044cab2d0aa\") " pod="odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh" Jul 18 01:56:55.938073 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.938036 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-xc626"] Jul 18 01:56:55.942161 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.942138 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-xc626" Jul 18 01:56:55.946343 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.946321 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 18 01:56:55.946457 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.946321 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-8mprl\"" Jul 18 01:56:55.951176 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.951152 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-xc626"] Jul 18 01:56:55.960277 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.960244 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-l2p85"] Jul 18 01:56:55.964772 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.964728 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-l2p85" Jul 18 01:56:55.978078 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.978052 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-l2p85"] Jul 18 01:56:55.993013 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.992983 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f603196-b92d-4b4f-adc1-34698e2cb4d0-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-xc626\" (UID: \"5f603196-b92d-4b4f-adc1-34698e2cb4d0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xc626" Jul 18 01:56:55.993013 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.993019 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42mf5\" (UniqueName: \"kubernetes.io/projected/5f603196-b92d-4b4f-adc1-34698e2cb4d0-kube-api-access-42mf5\") pod \"payload-pre-processing-7677b774bb-xc626\" (UID: \"5f603196-b92d-4b4f-adc1-34698e2cb4d0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xc626" Jul 18 01:56:55.993214 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.993106 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/807b01d3-a35e-40df-bbdc-7044cab2d0aa-maas-api-tls\") pod \"maas-api-5fc78b4c74-sbwsh\" (UID: \"807b01d3-a35e-40df-bbdc-7044cab2d0aa\") " pod="odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh" Jul 18 01:56:55.995600 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:55.995573 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/807b01d3-a35e-40df-bbdc-7044cab2d0aa-maas-api-tls\") pod \"maas-api-5fc78b4c74-sbwsh\" (UID: \"807b01d3-a35e-40df-bbdc-7044cab2d0aa\") " pod="odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh" Jul 18 01:56:56.094148 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.094100 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0fe1c0b-338a-437e-8f00-6d4c560a7067-plugins-config-volume\") pod \"payload-processing-5bf47cd988-l2p85\" (UID: \"d0fe1c0b-338a-437e-8f00-6d4c560a7067\") " pod="openshift-ingress/payload-processing-5bf47cd988-l2p85" Jul 18 01:56:56.094339 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.094167 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm9dj\" (UniqueName: \"kubernetes.io/projected/d0fe1c0b-338a-437e-8f00-6d4c560a7067-kube-api-access-vm9dj\") pod \"payload-processing-5bf47cd988-l2p85\" (UID: \"d0fe1c0b-338a-437e-8f00-6d4c560a7067\") " pod="openshift-ingress/payload-processing-5bf47cd988-l2p85" Jul 18 01:56:56.094339 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.094204 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f603196-b92d-4b4f-adc1-34698e2cb4d0-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-xc626\" (UID: \"5f603196-b92d-4b4f-adc1-34698e2cb4d0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xc626" Jul 18 01:56:56.094339 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.094230 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42mf5\" (UniqueName: \"kubernetes.io/projected/5f603196-b92d-4b4f-adc1-34698e2cb4d0-kube-api-access-42mf5\") pod \"payload-pre-processing-7677b774bb-xc626\" (UID: \"5f603196-b92d-4b4f-adc1-34698e2cb4d0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xc626" Jul 18 01:56:56.094871 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.094849 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5f603196-b92d-4b4f-adc1-34698e2cb4d0-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-xc626\" (UID: \"5f603196-b92d-4b4f-adc1-34698e2cb4d0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xc626" Jul 18 01:56:56.105128 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.105100 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42mf5\" (UniqueName: \"kubernetes.io/projected/5f603196-b92d-4b4f-adc1-34698e2cb4d0-kube-api-access-42mf5\") pod \"payload-pre-processing-7677b774bb-xc626\" (UID: \"5f603196-b92d-4b4f-adc1-34698e2cb4d0\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xc626" Jul 18 01:56:56.186041 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.186008 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh" Jul 18 01:56:56.195025 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.194998 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0fe1c0b-338a-437e-8f00-6d4c560a7067-plugins-config-volume\") pod \"payload-processing-5bf47cd988-l2p85\" (UID: \"d0fe1c0b-338a-437e-8f00-6d4c560a7067\") " pod="openshift-ingress/payload-processing-5bf47cd988-l2p85" Jul 18 01:56:56.195131 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.195051 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm9dj\" (UniqueName: \"kubernetes.io/projected/d0fe1c0b-338a-437e-8f00-6d4c560a7067-kube-api-access-vm9dj\") pod \"payload-processing-5bf47cd988-l2p85\" (UID: \"d0fe1c0b-338a-437e-8f00-6d4c560a7067\") " pod="openshift-ingress/payload-processing-5bf47cd988-l2p85" Jul 18 01:56:56.195651 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.195621 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0fe1c0b-338a-437e-8f00-6d4c560a7067-plugins-config-volume\") pod \"payload-processing-5bf47cd988-l2p85\" (UID: \"d0fe1c0b-338a-437e-8f00-6d4c560a7067\") " pod="openshift-ingress/payload-processing-5bf47cd988-l2p85" Jul 18 01:56:56.203240 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.203211 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm9dj\" (UniqueName: \"kubernetes.io/projected/d0fe1c0b-338a-437e-8f00-6d4c560a7067-kube-api-access-vm9dj\") pod \"payload-processing-5bf47cd988-l2p85\" (UID: \"d0fe1c0b-338a-437e-8f00-6d4c560a7067\") " pod="openshift-ingress/payload-processing-5bf47cd988-l2p85" Jul 18 01:56:56.255284 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.255168 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-xc626" Jul 18 01:56:56.277235 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.276931 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-l2p85" Jul 18 01:56:56.336278 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.336179 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh"] Jul 18 01:56:56.337971 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:56:56.337901 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod807b01d3_a35e_40df_bbdc_7044cab2d0aa.slice/crio-106c160bf7dfb0766fef0a97ef37b1cfe0bff83fd7be2bf3f8a540d77363f511 WatchSource:0}: Error finding container 106c160bf7dfb0766fef0a97ef37b1cfe0bff83fd7be2bf3f8a540d77363f511: Status 404 returned error can't find the container with id 106c160bf7dfb0766fef0a97ef37b1cfe0bff83fd7be2bf3f8a540d77363f511 Jul 18 01:56:56.424226 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.424020 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-xc626"] Jul 18 01:56:56.427355 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:56:56.427291 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f603196_b92d_4b4f_adc1_34698e2cb4d0.slice/crio-034e2b831e7f00fa5a19d830443fbc57450a2bb2a070b973ce72a697db6ba3ee WatchSource:0}: Error finding container 034e2b831e7f00fa5a19d830443fbc57450a2bb2a070b973ce72a697db6ba3ee: Status 404 returned error can't find the container with id 034e2b831e7f00fa5a19d830443fbc57450a2bb2a070b973ce72a697db6ba3ee Jul 18 01:56:56.470496 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.470473 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-l2p85"] Jul 18 01:56:56.472964 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:56:56.472926 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0fe1c0b_338a_437e_8f00_6d4c560a7067.slice/crio-f9be4104b8143757899ff8410b2ea33da00b8179e10faa458988a3f0094712df WatchSource:0}: Error finding container f9be4104b8143757899ff8410b2ea33da00b8179e10faa458988a3f0094712df: Status 404 returned error can't find the container with id f9be4104b8143757899ff8410b2ea33da00b8179e10faa458988a3f0094712df Jul 18 01:56:56.588270 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.588233 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-l2p85" event={"ID":"d0fe1c0b-338a-437e-8f00-6d4c560a7067","Type":"ContainerStarted","Data":"f9be4104b8143757899ff8410b2ea33da00b8179e10faa458988a3f0094712df"} Jul 18 01:56:56.589359 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.589332 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-xc626" event={"ID":"5f603196-b92d-4b4f-adc1-34698e2cb4d0","Type":"ContainerStarted","Data":"034e2b831e7f00fa5a19d830443fbc57450a2bb2a070b973ce72a697db6ba3ee"} Jul 18 01:56:56.590594 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:56:56.590572 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh" event={"ID":"807b01d3-a35e-40df-bbdc-7044cab2d0aa","Type":"ContainerStarted","Data":"106c160bf7dfb0766fef0a97ef37b1cfe0bff83fd7be2bf3f8a540d77363f511"} Jul 18 01:57:00.616888 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:00.616842 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-l2p85" event={"ID":"d0fe1c0b-338a-437e-8f00-6d4c560a7067","Type":"ContainerStarted","Data":"9ee8a9c86a362965ffbaa78ba6b7facfee6bd89fb30e35c14d0f00e29b7b21ff"} Jul 18 01:57:00.617548 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:00.616914 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-l2p85" Jul 18 01:57:00.618539 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:00.618517 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-xc626" event={"ID":"5f603196-b92d-4b4f-adc1-34698e2cb4d0","Type":"ContainerStarted","Data":"a3a56456ca664a014a878629f3eac8cf3488d21df9628defb14a94638d1ece0d"} Jul 18 01:57:00.618604 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:00.618589 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-xc626" Jul 18 01:57:00.619911 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:00.619891 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh" event={"ID":"807b01d3-a35e-40df-bbdc-7044cab2d0aa","Type":"ContainerStarted","Data":"c3ea97481b53d70786ce0b82dc45e7dc83a1fba1e293dee19c1b8074d5717c5e"} Jul 18 01:57:00.620033 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:00.620022 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh" Jul 18 01:57:00.633213 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:00.633164 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-l2p85" podStartSLOduration=2.286934352 podStartE2EDuration="5.633145608s" totalImagesPullingTime="3.346211256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:00.631923105 +0000 UTC m=+560.309347217" watchObservedRunningTime="2026-07-18 01:57:00.633145608 +0000 UTC m=+560.310569722" Jul 18 01:57:00.648403 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:00.648330 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-xc626" podStartSLOduration=2.220170873 podStartE2EDuration="5.64831672s" totalImagesPullingTime="3.428145847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:00.647706899 +0000 UTC m=+560.325131009" watchObservedRunningTime="2026-07-18 01:57:00.64831672 +0000 UTC m=+560.325740865" Jul 18 01:57:00.664662 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:00.664592 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh" podStartSLOduration=2.230017468 podStartE2EDuration="5.664575268s" totalImagesPullingTime="3.4345578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:00.662973632 +0000 UTC m=+560.340397782" watchObservedRunningTime="2026-07-18 01:57:00.664575268 +0000 UTC m=+560.341999378" Jul 18 01:57:03.639730 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:03.639696 2570 generic.go:356] "Generic (PLEG): container finished" podID="be7b9a46-e566-4a97-a560-4d0e070934fe" containerID="c0e2076d83b122484b5273d51d13c3a092e40f6a11f6a3cbaacdb848ed4df16c" exitCode=0 Jul 18 01:57:03.640189 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:03.639771 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" event={"ID":"be7b9a46-e566-4a97-a560-4d0e070934fe","Type":"ContainerDied","Data":"c0e2076d83b122484b5273d51d13c3a092e40f6a11f6a3cbaacdb848ed4df16c"} Jul 18 01:57:03.640189 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:03.640168 2570 scope.go:117] "RemoveContainer" containerID="c0e2076d83b122484b5273d51d13c3a092e40f6a11f6a3cbaacdb848ed4df16c" Jul 18 01:57:03.641193 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:03.641168 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:57:03.647136 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:03.641694 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:57:03.647136 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:03.641821 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:57:04.649515 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:04.649474 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" event={"ID":"be7b9a46-e566-4a97-a560-4d0e070934fe","Type":"ContainerStarted","Data":"c9b6a9b70bab1d36a0a1ebe52616356d0bd0b2083917a15cc02b60c9c62fbfd9"} Jul 18 01:57:04.805439 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:04.805407 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:57:04.805684 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:04.805666 2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-gf7sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 18 01:57:04.805743 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:04.805709 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" podUID="be7b9a46-e566-4a97-a560-4d0e070934fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 18 01:57:05.805299 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:05.805261 2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-gf7sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 18 01:57:05.805723 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:05.805319 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" podUID="be7b9a46-e566-4a97-a560-4d0e070934fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 18 01:57:06.650032 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:06.649994 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5fc78b4c74-sbwsh" Jul 18 01:57:06.805307 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:06.805273 2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-gf7sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 18 01:57:06.805739 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:06.805329 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" podUID="be7b9a46-e566-4a97-a560-4d0e070934fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 18 01:57:07.805743 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:07.805708 2570 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-gf7sm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 18 01:57:07.806158 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:07.805766 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" podUID="be7b9a46-e566-4a97-a560-4d0e070934fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 18 01:57:08.809331 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:08.809296 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:57:08.809818 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:08.809591 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:57:08.810445 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:08.810422 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-gf7sm" Jul 18 01:57:11.628819 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:11.628773 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-xc626" Jul 18 01:57:11.629296 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:11.628869 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-l2p85" Jul 18 01:57:14.667913 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.667879 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg"] Jul 18 01:57:14.671497 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.671475 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.675253 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.675232 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 18 01:57:14.675384 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.675277 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 18 01:57:14.675384 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.675238 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dghpp\"" Jul 18 01:57:14.675384 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.675336 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 18 01:57:14.682998 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.682970 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg"] Jul 18 01:57:14.794352 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.794311 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.794529 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.794360 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.794529 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.794387 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z269\" (UniqueName: \"kubernetes.io/projected/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-kube-api-access-9z269\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.794529 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.794423 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.794529 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.794464 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.794529 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.794488 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.895278 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.895247 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.895278 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.895277 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.895543 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.895295 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z269\" (UniqueName: \"kubernetes.io/projected/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-kube-api-access-9z269\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.895543 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.895322 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.895543 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.895351 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.895543 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.895366 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.895787 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.895740 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.895836 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.895787 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.895836 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.895816 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.897715 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.897692 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.897900 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.897884 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.903396 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.903376 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z269\" (UniqueName: \"kubernetes.io/projected/0524fea7-2bce-4d9d-bea9-9b6a4cec78a4-kube-api-access-9z269\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c6nqg\" (UID: \"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:14.983332 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:14.983234 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:15.119796 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:15.119772 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg"] Jul 18 01:57:15.122094 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:57:15.122068 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0524fea7_2bce_4d9d_bea9_9b6a4cec78a4.slice/crio-0308db24d3571cfafd6344645ee0cb28e053c951ea44c4321e573982e655c84d WatchSource:0}: Error finding container 0308db24d3571cfafd6344645ee0cb28e053c951ea44c4321e573982e655c84d: Status 404 returned error can't find the container with id 0308db24d3571cfafd6344645ee0cb28e053c951ea44c4321e573982e655c84d Jul 18 01:57:15.707159 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:15.707117 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" event={"ID":"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4","Type":"ContainerStarted","Data":"0308db24d3571cfafd6344645ee0cb28e053c951ea44c4321e573982e655c84d"} Jul 18 01:57:16.715073 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:16.715038 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" event={"ID":"0524fea7-2bce-4d9d-bea9-9b6a4cec78a4","Type":"ContainerStarted","Data":"68d1e9dc82bf13ef759f7f1f7f9c21249c57600abd3ca046f182eb2c676f78fa"} Jul 18 01:57:16.733771 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:16.733713 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" podStartSLOduration=1.324417023 podStartE2EDuration="2.733695047s" totalImagesPullingTime="1.409278024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:16.731737502 +0000 UTC m=+576.409161614" watchObservedRunningTime="2026-07-18 01:57:16.733695047 +0000 UTC m=+576.411119158" Jul 18 01:57:24.984335 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:24.984297 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:24.984814 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:24.984362 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:24.997127 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:24.997106 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:25.770264 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:25.770237 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c6nqg" Jul 18 01:57:40.815734 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:40.815706 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 01:57:40.821634 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:40.821609 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 01:57:40.840279 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:40.840254 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 01:57:40.845977 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:40.845958 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 01:57:43.490086 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.490054 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn"] Jul 18 01:57:43.493583 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.493567 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.495840 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.495819 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 18 01:57:43.502312 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.502290 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn"] Jul 18 01:57:43.535473 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.535447 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.535583 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.535478 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.535583 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.535501 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw84d\" (UniqueName: \"kubernetes.io/projected/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-kube-api-access-mw84d\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.535583 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.535568 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.535746 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.535603 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.535746 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.535619 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.636413 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.636378 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.636614 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.636440 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.636614 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.636460 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.636614 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.636515 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.636614 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.636535 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.636614 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.636557 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw84d\" (UniqueName: \"kubernetes.io/projected/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-kube-api-access-mw84d\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.636937 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.636899 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.636982 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.636955 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.636982 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.636968 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.638954 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.638927 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.639075 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.638995 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.644561 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.644537 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw84d\" (UniqueName: \"kubernetes.io/projected/306e5cc7-cef8-49fc-a09a-f7a9010c2e9f-kube-api-access-mw84d\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn\" (UID: \"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.804076 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.804042 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:43.941598 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:43.941571 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn"] Jul 18 01:57:43.943293 ip-10-0-132-151 kubenswrapper[2570]: W0718 01:57:43.943265 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306e5cc7_cef8_49fc_a09a_f7a9010c2e9f.slice/crio-2b71f0600a706213c3e32e9d75b94650686eb1bfb23104ddc834f4df96b42ae8 WatchSource:0}: Error finding container 2b71f0600a706213c3e32e9d75b94650686eb1bfb23104ddc834f4df96b42ae8: Status 404 returned error can't find the container with id 2b71f0600a706213c3e32e9d75b94650686eb1bfb23104ddc834f4df96b42ae8 Jul 18 01:57:44.862779 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:44.862743 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" event={"ID":"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f","Type":"ContainerStarted","Data":"0362c1b768a234f34eb3c1e3cb17fb3df211df5b64c5cc92870d5aa8e75ba35c"} Jul 18 01:57:44.863166 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:44.862831 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" event={"ID":"306e5cc7-cef8-49fc-a09a-f7a9010c2e9f","Type":"ContainerStarted","Data":"2b71f0600a706213c3e32e9d75b94650686eb1bfb23104ddc834f4df96b42ae8"} Jul 18 01:57:44.879210 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:44.879155 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" podStartSLOduration=1.7053038919999999 podStartE2EDuration="1.879137977s" totalImagesPullingTime="173.834085ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:44.877294139 +0000 UTC m=+604.554718251" watchObservedRunningTime="2026-07-18 01:57:44.879137977 +0000 UTC m=+604.556562118" Jul 18 01:57:53.804549 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:53.804509 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:53.805066 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:53.804598 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:53.817601 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:53.817567 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 01:57:53.918030 ip-10-0-132-151 kubenswrapper[2570]: I0718 01:57:53.917997 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn" Jul 18 02:00:00.146546 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:00.146457 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hpbt4"] Jul 18 02:00:00.150020 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:00.150003 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hpbt4" Jul 18 02:00:00.152547 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:00.152527 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5chqh\"" Jul 18 02:00:00.157672 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:00.157172 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hpbt4"] Jul 18 02:00:00.161044 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:00.161024 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hpbt4" Jul 18 02:00:00.296628 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:00.294269 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hpbt4"] Jul 18 02:00:00.299412 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:00:00.299380 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda07bb45f_f1e2_4c7a_b338_dc74b3f6a56a.slice/crio-2ce758523de8099c63ff87e1a0890338bd6a04a487a189b7622dedc6f2e7ffba WatchSource:0}: Error finding container 2ce758523de8099c63ff87e1a0890338bd6a04a487a189b7622dedc6f2e7ffba: Status 404 returned error can't find the container with id 2ce758523de8099c63ff87e1a0890338bd6a04a487a189b7622dedc6f2e7ffba Jul 18 02:00:00.301467 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:00.301449 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 02:00:00.526517 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:00.526432 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hpbt4" event={"ID":"a07bb45f-f1e2-4c7a-b338-dc74b3f6a56a","Type":"ContainerStarted","Data":"2ce758523de8099c63ff87e1a0890338bd6a04a487a189b7622dedc6f2e7ffba"} Jul 18 02:00:04.549505 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:04.549473 2570 generic.go:356] "Generic (PLEG): container finished" podID="a07bb45f-f1e2-4c7a-b338-dc74b3f6a56a" containerID="cb9b7cbb351074b097b3f51b7f7740c123a652d83ac99a932ca8a7860c54ebf2" exitCode=0 Jul 18 02:00:04.549972 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:04.549556 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hpbt4" event={"ID":"a07bb45f-f1e2-4c7a-b338-dc74b3f6a56a","Type":"ContainerDied","Data":"cb9b7cbb351074b097b3f51b7f7740c123a652d83ac99a932ca8a7860c54ebf2"} Jul 18 02:00:05.698280 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:05.698253 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hpbt4" Jul 18 02:00:06.560906 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:06.560879 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hpbt4" Jul 18 02:00:06.561088 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:06.560915 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hpbt4" event={"ID":"a07bb45f-f1e2-4c7a-b338-dc74b3f6a56a","Type":"ContainerDied","Data":"2ce758523de8099c63ff87e1a0890338bd6a04a487a189b7622dedc6f2e7ffba"} Jul 18 02:00:06.561088 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:06.560963 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ce758523de8099c63ff87e1a0890338bd6a04a487a189b7622dedc6f2e7ffba" Jul 18 02:00:11.464176 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:11.464145 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-cb6d88699-8fjgj"] Jul 18 02:00:11.464721 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:11.464702 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a07bb45f-f1e2-4c7a-b338-dc74b3f6a56a" containerName="cleanup" Jul 18 02:00:11.464784 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:11.464726 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a07bb45f-f1e2-4c7a-b338-dc74b3f6a56a" containerName="cleanup" Jul 18 02:00:11.464821 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:11.464807 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a07bb45f-f1e2-4c7a-b338-dc74b3f6a56a" containerName="cleanup" Jul 18 02:00:11.467683 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:11.467669 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb6d88699-8fjgj" Jul 18 02:00:11.470798 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:11.470780 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 18 02:00:11.470924 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:11.470805 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bsnwq\"" Jul 18 02:00:11.475082 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:11.474711 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cb6d88699-8fjgj"] Jul 18 02:00:11.584853 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:11.584822 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bcc8d07c-e198-425c-9b69-266fdf3740b3-tls-cert\") pod \"authorino-cb6d88699-8fjgj\" (UID: \"bcc8d07c-e198-425c-9b69-266fdf3740b3\") " pod="kuadrant-system/authorino-cb6d88699-8fjgj" Jul 18 02:00:11.585051 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:11.584918 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd8cx\" (UniqueName: \"kubernetes.io/projected/bcc8d07c-e198-425c-9b69-266fdf3740b3-kube-api-access-rd8cx\") pod \"authorino-cb6d88699-8fjgj\" (UID: \"bcc8d07c-e198-425c-9b69-266fdf3740b3\") " pod="kuadrant-system/authorino-cb6d88699-8fjgj" Jul 18 02:00:11.686030 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:11.685999 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bcc8d07c-e198-425c-9b69-266fdf3740b3-tls-cert\") pod \"authorino-cb6d88699-8fjgj\" (UID: \"bcc8d07c-e198-425c-9b69-266fdf3740b3\") " pod="kuadrant-system/authorino-cb6d88699-8fjgj" Jul 18 02:00:11.686208 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:11.686059 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd8cx\" (UniqueName: \"kubernetes.io/projected/bcc8d07c-e198-425c-9b69-266fdf3740b3-kube-api-access-rd8cx\") pod \"authorino-cb6d88699-8fjgj\" (UID: \"bcc8d07c-e198-425c-9b69-266fdf3740b3\") " pod="kuadrant-system/authorino-cb6d88699-8fjgj" Jul 18 02:00:11.688503 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:11.688480 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bcc8d07c-e198-425c-9b69-266fdf3740b3-tls-cert\") pod \"authorino-cb6d88699-8fjgj\" (UID: \"bcc8d07c-e198-425c-9b69-266fdf3740b3\") " pod="kuadrant-system/authorino-cb6d88699-8fjgj" Jul 18 02:00:11.693296 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:11.693276 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd8cx\" (UniqueName: \"kubernetes.io/projected/bcc8d07c-e198-425c-9b69-266fdf3740b3-kube-api-access-rd8cx\") pod \"authorino-cb6d88699-8fjgj\" (UID: \"bcc8d07c-e198-425c-9b69-266fdf3740b3\") " pod="kuadrant-system/authorino-cb6d88699-8fjgj" Jul 18 02:00:11.778457 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:11.778372 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb6d88699-8fjgj" Jul 18 02:00:11.920965 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:11.920943 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cb6d88699-8fjgj"] Jul 18 02:00:11.923042 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:00:11.923017 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbcc8d07c_e198_425c_9b69_266fdf3740b3.slice/crio-37bb87c8934ee4c51c430250437286031f660e56d81a4bfc7fa6b0016d36ee24 WatchSource:0}: Error finding container 37bb87c8934ee4c51c430250437286031f660e56d81a4bfc7fa6b0016d36ee24: Status 404 returned error can't find the container with id 37bb87c8934ee4c51c430250437286031f660e56d81a4bfc7fa6b0016d36ee24 Jul 18 02:00:12.590991 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:12.590946 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cb6d88699-8fjgj" event={"ID":"bcc8d07c-e198-425c-9b69-266fdf3740b3","Type":"ContainerStarted","Data":"2c093b2bb2c418266efa2473b1a88e74de06f3d053fc4b26d7ffb26488eb24cf"} Jul 18 02:00:12.591414 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:12.590998 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cb6d88699-8fjgj" event={"ID":"bcc8d07c-e198-425c-9b69-266fdf3740b3","Type":"ContainerStarted","Data":"37bb87c8934ee4c51c430250437286031f660e56d81a4bfc7fa6b0016d36ee24"} Jul 18 02:00:12.608972 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:12.608928 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-cb6d88699-8fjgj" podStartSLOduration=1.13213916 podStartE2EDuration="1.608910635s" totalImagesPullingTime="476.771475ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:00:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:00:12.607597152 +0000 UTC m=+752.285021332" watchObservedRunningTime="2026-07-18 02:00:12.608910635 +0000 UTC m=+752.286334782" Jul 18 02:00:58.115243 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.115207 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74bf56b8d6-ltm5j"] Jul 18 02:00:58.115891 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.115555 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" podUID="86161998-684e-4b4d-b23c-015a7581158d" containerName="manager" containerID="cri-o://6c87042c8880b9d3877afeb8aadf8a2e3daf41b665c072c500e53f60b0cc9ddc" gracePeriod=30 Jul 18 02:00:58.366790 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.366732 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" Jul 18 02:00:58.507771 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.507736 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dczr\" (UniqueName: \"kubernetes.io/projected/86161998-684e-4b4d-b23c-015a7581158d-kube-api-access-6dczr\") pod \"86161998-684e-4b4d-b23c-015a7581158d\" (UID: \"86161998-684e-4b4d-b23c-015a7581158d\") " Jul 18 02:00:58.507970 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.507853 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86161998-684e-4b4d-b23c-015a7581158d-webhook-cert\") pod \"86161998-684e-4b4d-b23c-015a7581158d\" (UID: \"86161998-684e-4b4d-b23c-015a7581158d\") " Jul 18 02:00:58.509941 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.509908 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86161998-684e-4b4d-b23c-015a7581158d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "86161998-684e-4b4d-b23c-015a7581158d" (UID: "86161998-684e-4b4d-b23c-015a7581158d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:00:58.510058 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.509917 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86161998-684e-4b4d-b23c-015a7581158d-kube-api-access-6dczr" (OuterVolumeSpecName: "kube-api-access-6dczr") pod "86161998-684e-4b4d-b23c-015a7581158d" (UID: "86161998-684e-4b4d-b23c-015a7581158d"). InnerVolumeSpecName "kube-api-access-6dczr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:00:58.609338 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.609302 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dczr\" (UniqueName: \"kubernetes.io/projected/86161998-684e-4b4d-b23c-015a7581158d-kube-api-access-6dczr\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:00:58.609338 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.609332 2570 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86161998-684e-4b4d-b23c-015a7581158d-webhook-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:00:58.832565 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.832530 2570 generic.go:356] "Generic (PLEG): container finished" podID="86161998-684e-4b4d-b23c-015a7581158d" containerID="6c87042c8880b9d3877afeb8aadf8a2e3daf41b665c072c500e53f60b0cc9ddc" exitCode=0 Jul 18 02:00:58.832800 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.832565 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" event={"ID":"86161998-684e-4b4d-b23c-015a7581158d","Type":"ContainerDied","Data":"6c87042c8880b9d3877afeb8aadf8a2e3daf41b665c072c500e53f60b0cc9ddc"} Jul 18 02:00:58.832800 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.832635 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" event={"ID":"86161998-684e-4b4d-b23c-015a7581158d","Type":"ContainerDied","Data":"49f69fe85bb13faf26e2fab2ee8c8ad33aba3808cbef8d895fff9e643ab4ad91"} Jul 18 02:00:58.832800 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.832574 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74bf56b8d6-ltm5j" Jul 18 02:00:58.832800 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.832681 2570 scope.go:117] "RemoveContainer" containerID="6c87042c8880b9d3877afeb8aadf8a2e3daf41b665c072c500e53f60b0cc9ddc" Jul 18 02:00:58.842166 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.842144 2570 scope.go:117] "RemoveContainer" containerID="6c87042c8880b9d3877afeb8aadf8a2e3daf41b665c072c500e53f60b0cc9ddc" Jul 18 02:00:58.842442 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:00:58.842419 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c87042c8880b9d3877afeb8aadf8a2e3daf41b665c072c500e53f60b0cc9ddc\": container with ID starting with 6c87042c8880b9d3877afeb8aadf8a2e3daf41b665c072c500e53f60b0cc9ddc not found: ID does not exist" containerID="6c87042c8880b9d3877afeb8aadf8a2e3daf41b665c072c500e53f60b0cc9ddc" Jul 18 02:00:58.842492 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.842453 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c87042c8880b9d3877afeb8aadf8a2e3daf41b665c072c500e53f60b0cc9ddc"} err="failed to get container status \"6c87042c8880b9d3877afeb8aadf8a2e3daf41b665c072c500e53f60b0cc9ddc\": rpc error: code = NotFound desc = could not find container \"6c87042c8880b9d3877afeb8aadf8a2e3daf41b665c072c500e53f60b0cc9ddc\": container with ID starting with 6c87042c8880b9d3877afeb8aadf8a2e3daf41b665c072c500e53f60b0cc9ddc not found: ID does not exist" Jul 18 02:00:58.853745 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.853721 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74bf56b8d6-ltm5j"] Jul 18 02:00:58.855531 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.855511 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74bf56b8d6-ltm5j"] Jul 18 02:00:58.894153 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:00:58.894130 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86161998-684e-4b4d-b23c-015a7581158d" path="/var/lib/kubelet/pods/86161998-684e-4b4d-b23c-015a7581158d/volumes" Jul 18 02:01:59.631747 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:01:59.631702 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74bf56b8d6-5vw4c"] Jul 18 02:01:59.632379 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:01:59.632084 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86161998-684e-4b4d-b23c-015a7581158d" containerName="manager" Jul 18 02:01:59.632379 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:01:59.632095 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="86161998-684e-4b4d-b23c-015a7581158d" containerName="manager" Jul 18 02:01:59.632379 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:01:59.632181 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="86161998-684e-4b4d-b23c-015a7581158d" containerName="manager" Jul 18 02:01:59.636402 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:01:59.636385 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" Jul 18 02:01:59.639145 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:01:59.639117 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 18 02:01:59.640135 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:01:59.640107 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 18 02:01:59.640278 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:01:59.640248 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nzhxz\"" Jul 18 02:01:59.643155 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:01:59.643131 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74bf56b8d6-5vw4c"] Jul 18 02:01:59.766560 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:01:59.766516 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c9j5\" (UniqueName: \"kubernetes.io/projected/7b7502ff-d1e4-4097-b004-9121f1009337-kube-api-access-2c9j5\") pod \"maas-controller-74bf56b8d6-5vw4c\" (UID: \"7b7502ff-d1e4-4097-b004-9121f1009337\") " pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" Jul 18 02:01:59.766771 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:01:59.766586 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b7502ff-d1e4-4097-b004-9121f1009337-webhook-cert\") pod \"maas-controller-74bf56b8d6-5vw4c\" (UID: \"7b7502ff-d1e4-4097-b004-9121f1009337\") " pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" Jul 18 02:01:59.867935 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:01:59.867893 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c9j5\" (UniqueName: \"kubernetes.io/projected/7b7502ff-d1e4-4097-b004-9121f1009337-kube-api-access-2c9j5\") pod \"maas-controller-74bf56b8d6-5vw4c\" (UID: \"7b7502ff-d1e4-4097-b004-9121f1009337\") " pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" Jul 18 02:01:59.868095 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:01:59.867969 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b7502ff-d1e4-4097-b004-9121f1009337-webhook-cert\") pod \"maas-controller-74bf56b8d6-5vw4c\" (UID: \"7b7502ff-d1e4-4097-b004-9121f1009337\") " pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" Jul 18 02:01:59.870277 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:01:59.870251 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b7502ff-d1e4-4097-b004-9121f1009337-webhook-cert\") pod \"maas-controller-74bf56b8d6-5vw4c\" (UID: \"7b7502ff-d1e4-4097-b004-9121f1009337\") " pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" Jul 18 02:01:59.875441 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:01:59.875417 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c9j5\" (UniqueName: \"kubernetes.io/projected/7b7502ff-d1e4-4097-b004-9121f1009337-kube-api-access-2c9j5\") pod \"maas-controller-74bf56b8d6-5vw4c\" (UID: \"7b7502ff-d1e4-4097-b004-9121f1009337\") " pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" Jul 18 02:01:59.950176 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:01:59.950120 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" Jul 18 02:02:00.084227 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:00.084200 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74bf56b8d6-5vw4c"] Jul 18 02:02:00.085949 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:02:00.085922 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b7502ff_d1e4_4097_b004_9121f1009337.slice/crio-1a79f845dccf05c15e23cc61f8f4b351f352713062ba44c4b26c077758b46c06 WatchSource:0}: Error finding container 1a79f845dccf05c15e23cc61f8f4b351f352713062ba44c4b26c077758b46c06: Status 404 returned error can't find the container with id 1a79f845dccf05c15e23cc61f8f4b351f352713062ba44c4b26c077758b46c06 Jul 18 02:02:00.131915 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:00.131879 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" event={"ID":"7b7502ff-d1e4-4097-b004-9121f1009337","Type":"ContainerStarted","Data":"1a79f845dccf05c15e23cc61f8f4b351f352713062ba44c4b26c077758b46c06"} Jul 18 02:02:01.139191 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:01.139102 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" event={"ID":"7b7502ff-d1e4-4097-b004-9121f1009337","Type":"ContainerStarted","Data":"c75787167a6f27fbfcfa59f6292621fa16c4b023a8b98c74a36455708d140e71"} Jul 18 02:02:01.139569 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:01.139220 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" Jul 18 02:02:01.165018 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:01.164826 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" podStartSLOduration=1.3737337809999999 podStartE2EDuration="2.164805531s" totalImagesPullingTime="791.07175ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:01:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:02:01.163322928 +0000 UTC m=+860.840747041" watchObservedRunningTime="2026-07-18 02:02:01.164805531 +0000 UTC m=+860.842229643" Jul 18 02:02:12.146989 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:12.146956 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" Jul 18 02:02:16.784217 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:16.784181 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74bf56b8d6-5vw4c"] Jul 18 02:02:16.784735 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:16.784393 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" podUID="7b7502ff-d1e4-4097-b004-9121f1009337" containerName="manager" containerID="cri-o://c75787167a6f27fbfcfa59f6292621fa16c4b023a8b98c74a36455708d140e71" gracePeriod=30 Jul 18 02:02:17.035841 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:17.035780 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" Jul 18 02:02:17.127893 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:17.127861 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b7502ff-d1e4-4097-b004-9121f1009337-webhook-cert\") pod \"7b7502ff-d1e4-4097-b004-9121f1009337\" (UID: \"7b7502ff-d1e4-4097-b004-9121f1009337\") " Jul 18 02:02:17.127893 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:17.127903 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2c9j5\" (UniqueName: \"kubernetes.io/projected/7b7502ff-d1e4-4097-b004-9121f1009337-kube-api-access-2c9j5\") pod \"7b7502ff-d1e4-4097-b004-9121f1009337\" (UID: \"7b7502ff-d1e4-4097-b004-9121f1009337\") " Jul 18 02:02:17.129937 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:17.129913 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b7502ff-d1e4-4097-b004-9121f1009337-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7b7502ff-d1e4-4097-b004-9121f1009337" (UID: "7b7502ff-d1e4-4097-b004-9121f1009337"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:02:17.130056 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:17.129963 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b7502ff-d1e4-4097-b004-9121f1009337-kube-api-access-2c9j5" (OuterVolumeSpecName: "kube-api-access-2c9j5") pod "7b7502ff-d1e4-4097-b004-9121f1009337" (UID: "7b7502ff-d1e4-4097-b004-9121f1009337"). InnerVolumeSpecName "kube-api-access-2c9j5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:02:17.220174 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:17.220141 2570 generic.go:356] "Generic (PLEG): container finished" podID="7b7502ff-d1e4-4097-b004-9121f1009337" containerID="c75787167a6f27fbfcfa59f6292621fa16c4b023a8b98c74a36455708d140e71" exitCode=0 Jul 18 02:02:17.220352 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:17.220184 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" Jul 18 02:02:17.220352 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:17.220192 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" event={"ID":"7b7502ff-d1e4-4097-b004-9121f1009337","Type":"ContainerDied","Data":"c75787167a6f27fbfcfa59f6292621fa16c4b023a8b98c74a36455708d140e71"} Jul 18 02:02:17.220352 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:17.220234 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74bf56b8d6-5vw4c" event={"ID":"7b7502ff-d1e4-4097-b004-9121f1009337","Type":"ContainerDied","Data":"1a79f845dccf05c15e23cc61f8f4b351f352713062ba44c4b26c077758b46c06"} Jul 18 02:02:17.220352 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:17.220255 2570 scope.go:117] "RemoveContainer" containerID="c75787167a6f27fbfcfa59f6292621fa16c4b023a8b98c74a36455708d140e71" Jul 18 02:02:17.228536 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:17.228513 2570 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b7502ff-d1e4-4097-b004-9121f1009337-webhook-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:02:17.228728 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:17.228712 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c9j5\" (UniqueName: \"kubernetes.io/projected/7b7502ff-d1e4-4097-b004-9121f1009337-kube-api-access-2c9j5\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:02:17.231703 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:17.231684 2570 scope.go:117] "RemoveContainer" containerID="c75787167a6f27fbfcfa59f6292621fa16c4b023a8b98c74a36455708d140e71" Jul 18 02:02:17.232091 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:02:17.232071 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c75787167a6f27fbfcfa59f6292621fa16c4b023a8b98c74a36455708d140e71\": container with ID starting with c75787167a6f27fbfcfa59f6292621fa16c4b023a8b98c74a36455708d140e71 not found: ID does not exist" containerID="c75787167a6f27fbfcfa59f6292621fa16c4b023a8b98c74a36455708d140e71" Jul 18 02:02:17.232169 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:17.232101 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c75787167a6f27fbfcfa59f6292621fa16c4b023a8b98c74a36455708d140e71"} err="failed to get container status \"c75787167a6f27fbfcfa59f6292621fa16c4b023a8b98c74a36455708d140e71\": rpc error: code = NotFound desc = could not find container \"c75787167a6f27fbfcfa59f6292621fa16c4b023a8b98c74a36455708d140e71\": container with ID starting with c75787167a6f27fbfcfa59f6292621fa16c4b023a8b98c74a36455708d140e71 not found: ID does not exist" Jul 18 02:02:17.241702 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:17.241681 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74bf56b8d6-5vw4c"] Jul 18 02:02:17.245102 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:17.245078 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74bf56b8d6-5vw4c"] Jul 18 02:02:18.013822 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.013789 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74bf56b8d6-jk6xr"] Jul 18 02:02:18.014201 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.014186 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b7502ff-d1e4-4097-b004-9121f1009337" containerName="manager" Jul 18 02:02:18.014201 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.014198 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b7502ff-d1e4-4097-b004-9121f1009337" containerName="manager" Jul 18 02:02:18.014271 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.014254 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b7502ff-d1e4-4097-b004-9121f1009337" containerName="manager" Jul 18 02:02:18.018551 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.018531 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74bf56b8d6-jk6xr" Jul 18 02:02:18.021390 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.021370 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 18 02:02:18.021530 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.021391 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 18 02:02:18.021530 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.021447 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-nzhxz\"" Jul 18 02:02:18.026389 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.026373 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74bf56b8d6-jk6xr"] Jul 18 02:02:18.137866 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.137826 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnshb\" (UniqueName: \"kubernetes.io/projected/02f9098a-81c5-4b4b-8eaf-0fc55cbe7519-kube-api-access-jnshb\") pod \"maas-controller-74bf56b8d6-jk6xr\" (UID: \"02f9098a-81c5-4b4b-8eaf-0fc55cbe7519\") " pod="opendatahub/maas-controller-74bf56b8d6-jk6xr" Jul 18 02:02:18.138055 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.137885 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02f9098a-81c5-4b4b-8eaf-0fc55cbe7519-webhook-cert\") pod \"maas-controller-74bf56b8d6-jk6xr\" (UID: \"02f9098a-81c5-4b4b-8eaf-0fc55cbe7519\") " pod="opendatahub/maas-controller-74bf56b8d6-jk6xr" Jul 18 02:02:18.238856 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.238821 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02f9098a-81c5-4b4b-8eaf-0fc55cbe7519-webhook-cert\") pod \"maas-controller-74bf56b8d6-jk6xr\" (UID: \"02f9098a-81c5-4b4b-8eaf-0fc55cbe7519\") " pod="opendatahub/maas-controller-74bf56b8d6-jk6xr" Jul 18 02:02:18.239067 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.238924 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnshb\" (UniqueName: \"kubernetes.io/projected/02f9098a-81c5-4b4b-8eaf-0fc55cbe7519-kube-api-access-jnshb\") pod \"maas-controller-74bf56b8d6-jk6xr\" (UID: \"02f9098a-81c5-4b4b-8eaf-0fc55cbe7519\") " pod="opendatahub/maas-controller-74bf56b8d6-jk6xr" Jul 18 02:02:18.241386 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.241359 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02f9098a-81c5-4b4b-8eaf-0fc55cbe7519-webhook-cert\") pod \"maas-controller-74bf56b8d6-jk6xr\" (UID: \"02f9098a-81c5-4b4b-8eaf-0fc55cbe7519\") " pod="opendatahub/maas-controller-74bf56b8d6-jk6xr" Jul 18 02:02:18.247125 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.247102 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnshb\" (UniqueName: \"kubernetes.io/projected/02f9098a-81c5-4b4b-8eaf-0fc55cbe7519-kube-api-access-jnshb\") pod \"maas-controller-74bf56b8d6-jk6xr\" (UID: \"02f9098a-81c5-4b4b-8eaf-0fc55cbe7519\") " pod="opendatahub/maas-controller-74bf56b8d6-jk6xr" Jul 18 02:02:18.330778 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.330744 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74bf56b8d6-jk6xr" Jul 18 02:02:18.463276 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.463253 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74bf56b8d6-jk6xr"] Jul 18 02:02:18.464958 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:02:18.464931 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02f9098a_81c5_4b4b_8eaf_0fc55cbe7519.slice/crio-646cbc896185967adf2bc0d4f7e7fcbad00c4d247f4cd840a828edbe8f133557 WatchSource:0}: Error finding container 646cbc896185967adf2bc0d4f7e7fcbad00c4d247f4cd840a828edbe8f133557: Status 404 returned error can't find the container with id 646cbc896185967adf2bc0d4f7e7fcbad00c4d247f4cd840a828edbe8f133557 Jul 18 02:02:18.894682 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:18.894632 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b7502ff-d1e4-4097-b004-9121f1009337" path="/var/lib/kubelet/pods/7b7502ff-d1e4-4097-b004-9121f1009337/volumes" Jul 18 02:02:19.232729 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:19.232627 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74bf56b8d6-jk6xr" event={"ID":"02f9098a-81c5-4b4b-8eaf-0fc55cbe7519","Type":"ContainerStarted","Data":"58011ca79763ef35dda1cf846e5abc5f5e8b945bb32fa75a0588aa28bb1ce467"} Jul 18 02:02:19.233096 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:19.232824 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74bf56b8d6-jk6xr" Jul 18 02:02:19.233096 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:19.232874 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74bf56b8d6-jk6xr" event={"ID":"02f9098a-81c5-4b4b-8eaf-0fc55cbe7519","Type":"ContainerStarted","Data":"646cbc896185967adf2bc0d4f7e7fcbad00c4d247f4cd840a828edbe8f133557"} Jul 18 02:02:19.251996 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:19.251952 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74bf56b8d6-jk6xr" podStartSLOduration=1.789707843 podStartE2EDuration="2.251935891s" totalImagesPullingTime="462.228048ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:02:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:02:19.250279087 +0000 UTC m=+878.927703198" watchObservedRunningTime="2026-07-18 02:02:19.251935891 +0000 UTC m=+878.929360038" Jul 18 02:02:30.239627 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:30.239593 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74bf56b8d6-jk6xr" Jul 18 02:02:40.848905 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:40.848839 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 02:02:40.860586 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:40.860555 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 02:02:40.875451 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:40.875427 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 02:02:40.882308 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:02:40.882290 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 02:07:14.908859 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:14.908824 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx"] Jul 18 02:07:14.911289 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:14.909006 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" podUID="aa5e9117-9d90-4255-a56a-52b9cf3b816b" containerName="manager" containerID="cri-o://3409dc81ecc0aff753be438e36d10947ca4314958f20b8cd0f782c14d63b28b3" gracePeriod=10 Jul 18 02:07:18.562348 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:18.562321 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" Jul 18 02:07:18.716461 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:18.716358 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbwcc\" (UniqueName: \"kubernetes.io/projected/aa5e9117-9d90-4255-a56a-52b9cf3b816b-kube-api-access-tbwcc\") pod \"aa5e9117-9d90-4255-a56a-52b9cf3b816b\" (UID: \"aa5e9117-9d90-4255-a56a-52b9cf3b816b\") " Jul 18 02:07:18.716461 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:18.716432 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa5e9117-9d90-4255-a56a-52b9cf3b816b-extensions-socket-volume\") pod \"aa5e9117-9d90-4255-a56a-52b9cf3b816b\" (UID: \"aa5e9117-9d90-4255-a56a-52b9cf3b816b\") " Jul 18 02:07:18.716861 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:18.716835 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa5e9117-9d90-4255-a56a-52b9cf3b816b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "aa5e9117-9d90-4255-a56a-52b9cf3b816b" (UID: "aa5e9117-9d90-4255-a56a-52b9cf3b816b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:07:18.718535 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:18.718503 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa5e9117-9d90-4255-a56a-52b9cf3b816b-kube-api-access-tbwcc" (OuterVolumeSpecName: "kube-api-access-tbwcc") pod "aa5e9117-9d90-4255-a56a-52b9cf3b816b" (UID: "aa5e9117-9d90-4255-a56a-52b9cf3b816b"). InnerVolumeSpecName "kube-api-access-tbwcc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:07:18.763397 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:18.763360 2570 generic.go:356] "Generic (PLEG): container finished" podID="aa5e9117-9d90-4255-a56a-52b9cf3b816b" containerID="3409dc81ecc0aff753be438e36d10947ca4314958f20b8cd0f782c14d63b28b3" exitCode=0 Jul 18 02:07:18.763566 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:18.763408 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" Jul 18 02:07:18.763566 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:18.763424 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" event={"ID":"aa5e9117-9d90-4255-a56a-52b9cf3b816b","Type":"ContainerDied","Data":"3409dc81ecc0aff753be438e36d10947ca4314958f20b8cd0f782c14d63b28b3"} Jul 18 02:07:18.763566 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:18.763477 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx" event={"ID":"aa5e9117-9d90-4255-a56a-52b9cf3b816b","Type":"ContainerDied","Data":"1ce8e947e983c42bb8dd135495a90176ac2a2e1d8cfdeaed252b84d0d244c6a6"} Jul 18 02:07:18.763566 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:18.763506 2570 scope.go:117] "RemoveContainer" containerID="3409dc81ecc0aff753be438e36d10947ca4314958f20b8cd0f782c14d63b28b3" Jul 18 02:07:18.773504 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:18.773487 2570 scope.go:117] "RemoveContainer" containerID="3409dc81ecc0aff753be438e36d10947ca4314958f20b8cd0f782c14d63b28b3" Jul 18 02:07:18.773824 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:07:18.773804 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3409dc81ecc0aff753be438e36d10947ca4314958f20b8cd0f782c14d63b28b3\": container with ID starting with 3409dc81ecc0aff753be438e36d10947ca4314958f20b8cd0f782c14d63b28b3 not found: ID does not exist" containerID="3409dc81ecc0aff753be438e36d10947ca4314958f20b8cd0f782c14d63b28b3" Jul 18 02:07:18.773896 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:18.773832 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3409dc81ecc0aff753be438e36d10947ca4314958f20b8cd0f782c14d63b28b3"} err="failed to get container status \"3409dc81ecc0aff753be438e36d10947ca4314958f20b8cd0f782c14d63b28b3\": rpc error: code = NotFound desc = could not find container \"3409dc81ecc0aff753be438e36d10947ca4314958f20b8cd0f782c14d63b28b3\": container with ID starting with 3409dc81ecc0aff753be438e36d10947ca4314958f20b8cd0f782c14d63b28b3 not found: ID does not exist" Jul 18 02:07:18.791717 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:18.785673 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx"] Jul 18 02:07:18.791717 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:18.791593 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fjxwx"] Jul 18 02:07:18.817807 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:18.817772 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbwcc\" (UniqueName: \"kubernetes.io/projected/aa5e9117-9d90-4255-a56a-52b9cf3b816b-kube-api-access-tbwcc\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:07:18.817807 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:18.817806 2570 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa5e9117-9d90-4255-a56a-52b9cf3b816b-extensions-socket-volume\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:07:18.894774 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:18.894742 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa5e9117-9d90-4255-a56a-52b9cf3b816b" path="/var/lib/kubelet/pods/aa5e9117-9d90-4255-a56a-52b9cf3b816b/volumes" Jul 18 02:07:40.886549 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:40.886511 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 02:07:40.893565 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:40.893541 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 02:07:40.905315 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:40.905292 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 02:07:40.911059 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:07:40.911037 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 02:08:16.045420 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:16.045376 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dkwx8"] Jul 18 02:08:16.046056 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:16.045975 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa5e9117-9d90-4255-a56a-52b9cf3b816b" containerName="manager" Jul 18 02:08:16.046056 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:16.045995 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa5e9117-9d90-4255-a56a-52b9cf3b816b" containerName="manager" Jul 18 02:08:16.049716 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:16.046771 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa5e9117-9d90-4255-a56a-52b9cf3b816b" containerName="manager" Jul 18 02:08:16.051932 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:16.051909 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dkwx8" Jul 18 02:08:16.055122 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:16.055100 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4g74j\"" Jul 18 02:08:16.063101 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:16.063081 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dkwx8"] Jul 18 02:08:16.243846 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:16.243803 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5faad5dd-db71-43f3-8a67-2bc9b7555c67-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dkwx8\" (UID: \"5faad5dd-db71-43f3-8a67-2bc9b7555c67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dkwx8" Jul 18 02:08:16.243846 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:16.243847 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6g4q\" (UniqueName: \"kubernetes.io/projected/5faad5dd-db71-43f3-8a67-2bc9b7555c67-kube-api-access-t6g4q\") pod \"kuadrant-operator-controller-manager-86878c769f-dkwx8\" (UID: \"5faad5dd-db71-43f3-8a67-2bc9b7555c67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dkwx8" Jul 18 02:08:16.344878 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:16.344836 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5faad5dd-db71-43f3-8a67-2bc9b7555c67-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dkwx8\" (UID: \"5faad5dd-db71-43f3-8a67-2bc9b7555c67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dkwx8" Jul 18 02:08:16.345085 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:16.344884 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6g4q\" (UniqueName: \"kubernetes.io/projected/5faad5dd-db71-43f3-8a67-2bc9b7555c67-kube-api-access-t6g4q\") pod \"kuadrant-operator-controller-manager-86878c769f-dkwx8\" (UID: \"5faad5dd-db71-43f3-8a67-2bc9b7555c67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dkwx8" Jul 18 02:08:16.345237 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:16.345213 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5faad5dd-db71-43f3-8a67-2bc9b7555c67-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-dkwx8\" (UID: \"5faad5dd-db71-43f3-8a67-2bc9b7555c67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dkwx8" Jul 18 02:08:16.359787 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:16.359758 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6g4q\" (UniqueName: \"kubernetes.io/projected/5faad5dd-db71-43f3-8a67-2bc9b7555c67-kube-api-access-t6g4q\") pod \"kuadrant-operator-controller-manager-86878c769f-dkwx8\" (UID: \"5faad5dd-db71-43f3-8a67-2bc9b7555c67\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dkwx8" Jul 18 02:08:16.363565 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:16.363549 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dkwx8" Jul 18 02:08:16.511158 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:16.511132 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dkwx8"] Jul 18 02:08:16.513587 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:08:16.513561 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5faad5dd_db71_43f3_8a67_2bc9b7555c67.slice/crio-1dc60fad3d6a71b1f0ef7b83465902527d7bba253781ee6e235ce51c55e2f1b3 WatchSource:0}: Error finding container 1dc60fad3d6a71b1f0ef7b83465902527d7bba253781ee6e235ce51c55e2f1b3: Status 404 returned error can't find the container with id 1dc60fad3d6a71b1f0ef7b83465902527d7bba253781ee6e235ce51c55e2f1b3 Jul 18 02:08:17.051987 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:17.051955 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dkwx8" event={"ID":"5faad5dd-db71-43f3-8a67-2bc9b7555c67","Type":"ContainerStarted","Data":"6084bec9b0b3c2fc38cd1371dec95fde75b8c4b207a457d30758ee14e435ddb7"} Jul 18 02:08:17.052388 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:17.052006 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dkwx8" event={"ID":"5faad5dd-db71-43f3-8a67-2bc9b7555c67","Type":"ContainerStarted","Data":"1dc60fad3d6a71b1f0ef7b83465902527d7bba253781ee6e235ce51c55e2f1b3"} Jul 18 02:08:17.052388 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:17.052059 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dkwx8" Jul 18 02:08:17.076258 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:17.076199 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dkwx8" podStartSLOduration=1.07618272 podStartE2EDuration="1.07618272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:08:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:08:17.07301375 +0000 UTC m=+1236.750437862" watchObservedRunningTime="2026-07-18 02:08:17.07618272 +0000 UTC m=+1236.753606831" Jul 18 02:08:28.059358 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:08:28.059329 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-dkwx8" Jul 18 02:12:40.922742 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:12:40.922714 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 02:12:40.930400 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:12:40.930371 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 02:12:40.939161 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:12:40.939135 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 02:12:40.946034 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:12:40.946007 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 02:14:41.011798 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.011710 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt"] Jul 18 02:14:41.015679 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.015656 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.017971 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.017949 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a4dbbda0-openshift-default-dockercfg-zxc4z\"" Jul 18 02:14:41.034953 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.034918 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt"] Jul 18 02:14:41.160732 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.160700 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-workload-socket\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.160929 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.160739 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b4ab932-1d25-467c-982b-6d915da5c228-istio-podinfo\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.160929 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.160770 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-workload-certs\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.160929 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.160831 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-istio-envoy\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.160929 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.160913 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b4ab932-1d25-467c-982b-6d915da5c228-istio-token\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.161098 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.160970 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v8xv\" (UniqueName: \"kubernetes.io/projected/1b4ab932-1d25-467c-982b-6d915da5c228-kube-api-access-4v8xv\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.161098 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.161014 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-credential-socket\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.161098 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.161036 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-istio-data\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.161098 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.161058 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b4ab932-1d25-467c-982b-6d915da5c228-istiod-ca-cert\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.261535 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.261505 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v8xv\" (UniqueName: \"kubernetes.io/projected/1b4ab932-1d25-467c-982b-6d915da5c228-kube-api-access-4v8xv\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.261769 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.261550 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-credential-socket\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.261769 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.261578 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-istio-data\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.261769 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.261607 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b4ab932-1d25-467c-982b-6d915da5c228-istiod-ca-cert\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.261769 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.261679 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-workload-socket\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.261769 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.261704 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b4ab932-1d25-467c-982b-6d915da5c228-istio-podinfo\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.261769 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.261728 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-workload-certs\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.262142 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.261758 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-istio-envoy\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.262142 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.261929 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b4ab932-1d25-467c-982b-6d915da5c228-istio-token\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.262142 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.262018 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-credential-socket\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.262323 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.262186 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-workload-certs\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.262546 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.262499 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b4ab932-1d25-467c-982b-6d915da5c228-istiod-ca-cert\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.262669 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.262549 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-workload-socket\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.262861 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.262840 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-istio-data\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.264653 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.264614 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-istio-envoy\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.265112 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.265081 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b4ab932-1d25-467c-982b-6d915da5c228-istio-podinfo\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.270395 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.270366 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b4ab932-1d25-467c-982b-6d915da5c228-istio-token\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.270569 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.270501 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v8xv\" (UniqueName: \"kubernetes.io/projected/1b4ab932-1d25-467c-982b-6d915da5c228-kube-api-access-4v8xv\") pod \"e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.330695 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.330654 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:41.483760 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.483730 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt"] Jul 18 02:14:41.485210 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:14:41.485185 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b4ab932_1d25_467c_982b_6d915da5c228.slice/crio-990c8f7c28827dd34cae1c778cb743668833797c165c71e4f4d6e951d4a91c1a WatchSource:0}: Error finding container 990c8f7c28827dd34cae1c778cb743668833797c165c71e4f4d6e951d4a91c1a: Status 404 returned error can't find the container with id 990c8f7c28827dd34cae1c778cb743668833797c165c71e4f4d6e951d4a91c1a Jul 18 02:14:41.487437 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.487404 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:14:41.487519 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.487469 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:14:41.487519 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:41.487509 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:14:42.059626 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:42.059587 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" event={"ID":"1b4ab932-1d25-467c-982b-6d915da5c228","Type":"ContainerStarted","Data":"4e10655534d98e80f18ddf67779909e95bc1e05ae110e0a468db02eaf57cb3d5"} Jul 18 02:14:42.060045 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:42.059660 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" event={"ID":"1b4ab932-1d25-467c-982b-6d915da5c228","Type":"ContainerStarted","Data":"990c8f7c28827dd34cae1c778cb743668833797c165c71e4f4d6e951d4a91c1a"} Jul 18 02:14:42.084829 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:42.084773 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" podStartSLOduration=2.084758921 podStartE2EDuration="2.084758921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:14:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:14:42.084435635 +0000 UTC m=+1621.761859740" watchObservedRunningTime="2026-07-18 02:14:42.084758921 +0000 UTC m=+1621.762183032" Jul 18 02:14:42.331103 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:42.331016 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:42.336970 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:42.336940 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:43.065838 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:43.065785 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:43.066759 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:43.066740 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:54.130535 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:54.130499 2570 generic.go:356] "Generic (PLEG): container finished" podID="1b4ab932-1d25-467c-982b-6d915da5c228" containerID="4e10655534d98e80f18ddf67779909e95bc1e05ae110e0a468db02eaf57cb3d5" exitCode=0 Jul 18 02:14:54.130968 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:54.130573 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" event={"ID":"1b4ab932-1d25-467c-982b-6d915da5c228","Type":"ContainerDied","Data":"4e10655534d98e80f18ddf67779909e95bc1e05ae110e0a468db02eaf57cb3d5"} Jul 18 02:14:54.131017 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:54.130987 2570 scope.go:117] "RemoveContainer" containerID="4e10655534d98e80f18ddf67779909e95bc1e05ae110e0a468db02eaf57cb3d5" Jul 18 02:14:54.132050 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:54.132020 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:14:54.132150 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:54.132108 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:14:54.132213 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:54.132154 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:14:55.142220 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:55.142172 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" event={"ID":"1b4ab932-1d25-467c-982b-6d915da5c228","Type":"ContainerStarted","Data":"8eb30e9d49761a1600371cf75e88bff54c9d3de238fccca32eff1c0f3ece80fb"} Jul 18 02:14:55.331426 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:55.331381 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:55.333051 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:55.333014 2570 patch_prober.go:28] interesting pod/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 18 02:14:55.333195 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:55.333086 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" podUID="1b4ab932-1d25-467c-982b-6d915da5c228" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 18 02:14:56.331027 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:56.330983 2570 patch_prober.go:28] interesting pod/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 18 02:14:56.331522 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:56.331059 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" podUID="1b4ab932-1d25-467c-982b-6d915da5c228" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 18 02:14:57.331667 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:57.331599 2570 patch_prober.go:28] interesting pod/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 18 02:14:57.332185 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:57.331704 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" podUID="1b4ab932-1d25-467c-982b-6d915da5c228" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 18 02:14:58.066441 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:58.066407 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:58.331458 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:58.331355 2570 patch_prober.go:28] interesting pod/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 18 02:14:58.331458 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:58.331424 2570 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" podUID="1b4ab932-1d25-467c-982b-6d915da5c228" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 18 02:14:59.334967 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:59.334939 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:14:59.335911 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:14:59.335894 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:15:24.272546 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:24.272511 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt"] Jul 18 02:15:24.273185 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:24.272754 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" podUID="1b4ab932-1d25-467c-982b-6d915da5c228" containerName="istio-proxy" containerID="cri-o://8eb30e9d49761a1600371cf75e88bff54c9d3de238fccca32eff1c0f3ece80fb" gracePeriod=30 Jul 18 02:15:29.337284 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.337248 2570 patch_prober.go:28] interesting pod/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 18 02:15:29.337668 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.337315 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" podUID="1b4ab932-1d25-467c-982b-6d915da5c228" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 18 02:15:29.538197 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.538134 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:15:29.728204 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.728159 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-istio-envoy\") pod \"1b4ab932-1d25-467c-982b-6d915da5c228\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " Jul 18 02:15:29.728410 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.728245 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-credential-socket\") pod \"1b4ab932-1d25-467c-982b-6d915da5c228\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " Jul 18 02:15:29.728410 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.728282 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-workload-certs\") pod \"1b4ab932-1d25-467c-982b-6d915da5c228\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " Jul 18 02:15:29.728410 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.728320 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4v8xv\" (UniqueName: \"kubernetes.io/projected/1b4ab932-1d25-467c-982b-6d915da5c228-kube-api-access-4v8xv\") pod \"1b4ab932-1d25-467c-982b-6d915da5c228\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " Jul 18 02:15:29.728410 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.728391 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-workload-socket\") pod \"1b4ab932-1d25-467c-982b-6d915da5c228\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " Jul 18 02:15:29.728634 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.728428 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-istio-data\") pod \"1b4ab932-1d25-467c-982b-6d915da5c228\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " Jul 18 02:15:29.728634 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.728461 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b4ab932-1d25-467c-982b-6d915da5c228-istio-podinfo\") pod \"1b4ab932-1d25-467c-982b-6d915da5c228\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " Jul 18 02:15:29.728634 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.728484 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b4ab932-1d25-467c-982b-6d915da5c228-istio-token\") pod \"1b4ab932-1d25-467c-982b-6d915da5c228\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " Jul 18 02:15:29.728634 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.728510 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b4ab932-1d25-467c-982b-6d915da5c228-istiod-ca-cert\") pod \"1b4ab932-1d25-467c-982b-6d915da5c228\" (UID: \"1b4ab932-1d25-467c-982b-6d915da5c228\") " Jul 18 02:15:29.728634 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.728622 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1b4ab932-1d25-467c-982b-6d915da5c228" (UID: "1b4ab932-1d25-467c-982b-6d915da5c228"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:15:29.729195 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.728942 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1b4ab932-1d25-467c-982b-6d915da5c228" (UID: "1b4ab932-1d25-467c-982b-6d915da5c228"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:15:29.729195 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.728949 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b4ab932-1d25-467c-982b-6d915da5c228-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1b4ab932-1d25-467c-982b-6d915da5c228" (UID: "1b4ab932-1d25-467c-982b-6d915da5c228"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:15:29.729195 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.728661 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1b4ab932-1d25-467c-982b-6d915da5c228" (UID: "1b4ab932-1d25-467c-982b-6d915da5c228"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:15:29.729195 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.729192 2570 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-credential-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:15:29.729441 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.729212 2570 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-workload-certs\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:15:29.729441 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.729225 2570 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-workload-socket\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:15:29.729441 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.729237 2570 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b4ab932-1d25-467c-982b-6d915da5c228-istiod-ca-cert\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:15:29.729610 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.729561 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-istio-data" (OuterVolumeSpecName: "istio-data") pod "1b4ab932-1d25-467c-982b-6d915da5c228" (UID: "1b4ab932-1d25-467c-982b-6d915da5c228"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:15:29.731109 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.731085 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1b4ab932-1d25-467c-982b-6d915da5c228-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1b4ab932-1d25-467c-982b-6d915da5c228" (UID: "1b4ab932-1d25-467c-982b-6d915da5c228"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:15:29.731205 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.731105 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1b4ab932-1d25-467c-982b-6d915da5c228" (UID: "1b4ab932-1d25-467c-982b-6d915da5c228"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:15:29.731616 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.731596 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b4ab932-1d25-467c-982b-6d915da5c228-istio-token" (OuterVolumeSpecName: "istio-token") pod "1b4ab932-1d25-467c-982b-6d915da5c228" (UID: "1b4ab932-1d25-467c-982b-6d915da5c228"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:15:29.731710 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.731692 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b4ab932-1d25-467c-982b-6d915da5c228-kube-api-access-4v8xv" (OuterVolumeSpecName: "kube-api-access-4v8xv") pod "1b4ab932-1d25-467c-982b-6d915da5c228" (UID: "1b4ab932-1d25-467c-982b-6d915da5c228"). InnerVolumeSpecName "kube-api-access-4v8xv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:15:29.829896 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.829858 2570 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-istio-data\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:15:29.829896 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.829889 2570 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b4ab932-1d25-467c-982b-6d915da5c228-istio-podinfo\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:15:29.829896 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.829897 2570 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b4ab932-1d25-467c-982b-6d915da5c228-istio-token\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:15:29.829896 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.829905 2570 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b4ab932-1d25-467c-982b-6d915da5c228-istio-envoy\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:15:29.830172 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:29.829913 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4v8xv\" (UniqueName: \"kubernetes.io/projected/1b4ab932-1d25-467c-982b-6d915da5c228-kube-api-access-4v8xv\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:15:30.328657 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:30.328613 2570 generic.go:356] "Generic (PLEG): container finished" podID="1b4ab932-1d25-467c-982b-6d915da5c228" containerID="8eb30e9d49761a1600371cf75e88bff54c9d3de238fccca32eff1c0f3ece80fb" exitCode=0 Jul 18 02:15:30.328855 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:30.328691 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" Jul 18 02:15:30.328855 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:30.328721 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" event={"ID":"1b4ab932-1d25-467c-982b-6d915da5c228","Type":"ContainerDied","Data":"8eb30e9d49761a1600371cf75e88bff54c9d3de238fccca32eff1c0f3ece80fb"} Jul 18 02:15:30.328855 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:30.328802 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt" event={"ID":"1b4ab932-1d25-467c-982b-6d915da5c228","Type":"ContainerDied","Data":"990c8f7c28827dd34cae1c778cb743668833797c165c71e4f4d6e951d4a91c1a"} Jul 18 02:15:30.328855 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:30.328840 2570 scope.go:117] "RemoveContainer" containerID="8eb30e9d49761a1600371cf75e88bff54c9d3de238fccca32eff1c0f3ece80fb" Jul 18 02:15:30.339026 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:30.339006 2570 scope.go:117] "RemoveContainer" containerID="4e10655534d98e80f18ddf67779909e95bc1e05ae110e0a468db02eaf57cb3d5" Jul 18 02:15:30.347850 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:30.347829 2570 scope.go:117] "RemoveContainer" containerID="8eb30e9d49761a1600371cf75e88bff54c9d3de238fccca32eff1c0f3ece80fb" Jul 18 02:15:30.348171 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:15:30.348146 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eb30e9d49761a1600371cf75e88bff54c9d3de238fccca32eff1c0f3ece80fb\": container with ID starting with 8eb30e9d49761a1600371cf75e88bff54c9d3de238fccca32eff1c0f3ece80fb not found: ID does not exist" containerID="8eb30e9d49761a1600371cf75e88bff54c9d3de238fccca32eff1c0f3ece80fb" Jul 18 02:15:30.348220 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:30.348187 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eb30e9d49761a1600371cf75e88bff54c9d3de238fccca32eff1c0f3ece80fb"} err="failed to get container status \"8eb30e9d49761a1600371cf75e88bff54c9d3de238fccca32eff1c0f3ece80fb\": rpc error: code = NotFound desc = could not find container \"8eb30e9d49761a1600371cf75e88bff54c9d3de238fccca32eff1c0f3ece80fb\": container with ID starting with 8eb30e9d49761a1600371cf75e88bff54c9d3de238fccca32eff1c0f3ece80fb not found: ID does not exist" Jul 18 02:15:30.348220 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:30.348212 2570 scope.go:117] "RemoveContainer" containerID="4e10655534d98e80f18ddf67779909e95bc1e05ae110e0a468db02eaf57cb3d5" Jul 18 02:15:30.348509 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:15:30.348491 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e10655534d98e80f18ddf67779909e95bc1e05ae110e0a468db02eaf57cb3d5\": container with ID starting with 4e10655534d98e80f18ddf67779909e95bc1e05ae110e0a468db02eaf57cb3d5 not found: ID does not exist" containerID="4e10655534d98e80f18ddf67779909e95bc1e05ae110e0a468db02eaf57cb3d5" Jul 18 02:15:30.348575 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:30.348513 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e10655534d98e80f18ddf67779909e95bc1e05ae110e0a468db02eaf57cb3d5"} err="failed to get container status \"4e10655534d98e80f18ddf67779909e95bc1e05ae110e0a468db02eaf57cb3d5\": rpc error: code = NotFound desc = could not find container \"4e10655534d98e80f18ddf67779909e95bc1e05ae110e0a468db02eaf57cb3d5\": container with ID starting with 4e10655534d98e80f18ddf67779909e95bc1e05ae110e0a468db02eaf57cb3d5 not found: ID does not exist" Jul 18 02:15:30.349490 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:30.349451 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt"] Jul 18 02:15:30.352304 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:30.352280 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a4dbbda0-openshift-default-5ffd59f9-5cqxt"] Jul 18 02:15:30.897611 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:30.897573 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b4ab932-1d25-467c-982b-6d915da5c228" path="/var/lib/kubelet/pods/1b4ab932-1d25-467c-982b-6d915da5c228/volumes" Jul 18 02:15:31.087559 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.087522 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd"] Jul 18 02:15:31.087983 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.087968 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b4ab932-1d25-467c-982b-6d915da5c228" containerName="istio-proxy" Jul 18 02:15:31.088029 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.087986 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b4ab932-1d25-467c-982b-6d915da5c228" containerName="istio-proxy" Jul 18 02:15:31.088075 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.088066 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b4ab932-1d25-467c-982b-6d915da5c228" containerName="istio-proxy" Jul 18 02:15:31.088108 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.088078 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b4ab932-1d25-467c-982b-6d915da5c228" containerName="istio-proxy" Jul 18 02:15:31.092519 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.092499 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" Jul 18 02:15:31.094573 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.094551 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-d45c47d8-serving-cert\"" Jul 18 02:15:31.101257 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.101230 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd"] Jul 18 02:15:31.141930 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.141887 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gl7r\" (UniqueName: \"kubernetes.io/projected/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-kube-api-access-8gl7r\") pod \"maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd\" (UID: \"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" Jul 18 02:15:31.142110 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.141985 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-maas-api-tls\") pod \"maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd\" (UID: \"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" Jul 18 02:15:31.142110 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.142023 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-openshift-service-ca\") pod \"maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd\" (UID: \"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" Jul 18 02:15:31.243167 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.243060 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gl7r\" (UniqueName: \"kubernetes.io/projected/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-kube-api-access-8gl7r\") pod \"maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd\" (UID: \"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" Jul 18 02:15:31.243167 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.243118 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-maas-api-tls\") pod \"maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd\" (UID: \"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" Jul 18 02:15:31.243167 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.243145 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-openshift-service-ca\") pod \"maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd\" (UID: \"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" Jul 18 02:15:31.243896 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.243877 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-openshift-service-ca\") pod \"maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd\" (UID: \"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" Jul 18 02:15:31.245733 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.245716 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-maas-api-tls\") pod \"maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd\" (UID: \"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" Jul 18 02:15:31.250627 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.250600 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gl7r\" (UniqueName: \"kubernetes.io/projected/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-kube-api-access-8gl7r\") pod \"maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd\" (UID: \"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" Jul 18 02:15:31.405837 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.405801 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" Jul 18 02:15:31.553686 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.553625 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd"] Jul 18 02:15:31.555729 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:15:31.555705 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60d2b755_b6a5_4ea2_9f7c_de47ebd045ee.slice/crio-6dda427d6850ee0d02ca508e30f2249b1ffbb12974bf58f956cd805688093949 WatchSource:0}: Error finding container 6dda427d6850ee0d02ca508e30f2249b1ffbb12974bf58f956cd805688093949: Status 404 returned error can't find the container with id 6dda427d6850ee0d02ca508e30f2249b1ffbb12974bf58f956cd805688093949 Jul 18 02:15:31.557047 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:31.557030 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 02:15:32.343631 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:32.343577 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" event={"ID":"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee","Type":"ContainerStarted","Data":"345a5a84f7ce2f36f8c6c55a9124fee2ab45bd5cc4264b4654f49380cf86c913"} Jul 18 02:15:32.343862 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:32.343799 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" Jul 18 02:15:32.343862 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:32.343829 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" event={"ID":"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee","Type":"ContainerStarted","Data":"6dda427d6850ee0d02ca508e30f2249b1ffbb12974bf58f956cd805688093949"} Jul 18 02:15:32.362510 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:32.362440 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" podStartSLOduration=0.933619618 podStartE2EDuration="1.362418699s" totalImagesPullingTime="428.799081ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:15:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:15:32.359983869 +0000 UTC m=+1672.037407981" watchObservedRunningTime="2026-07-18 02:15:32.362418699 +0000 UTC m=+1672.039842811" Jul 18 02:15:38.355150 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:15:38.355113 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" Jul 18 02:16:06.295223 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:06.295135 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd"] Jul 18 02:16:06.295815 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:06.295391 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" podUID="60d2b755-b6a5-4ea2-9f7c-de47ebd045ee" containerName="maas-api" containerID="cri-o://345a5a84f7ce2f36f8c6c55a9124fee2ab45bd5cc4264b4654f49380cf86c913" gracePeriod=30 Jul 18 02:16:06.520408 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:06.520364 2570 generic.go:356] "Generic (PLEG): container finished" podID="60d2b755-b6a5-4ea2-9f7c-de47ebd045ee" containerID="345a5a84f7ce2f36f8c6c55a9124fee2ab45bd5cc4264b4654f49380cf86c913" exitCode=0 Jul 18 02:16:06.520580 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:06.520421 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" event={"ID":"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee","Type":"ContainerDied","Data":"345a5a84f7ce2f36f8c6c55a9124fee2ab45bd5cc4264b4654f49380cf86c913"} Jul 18 02:16:06.547700 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:06.547680 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" Jul 18 02:16:06.672688 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:06.672637 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-maas-api-tls\") pod \"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee\" (UID: \"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee\") " Jul 18 02:16:06.672875 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:06.672737 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8gl7r\" (UniqueName: \"kubernetes.io/projected/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-kube-api-access-8gl7r\") pod \"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee\" (UID: \"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee\") " Jul 18 02:16:06.672875 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:06.672850 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-openshift-service-ca\") pod \"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee\" (UID: \"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee\") " Jul 18 02:16:06.673255 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:06.673228 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "60d2b755-b6a5-4ea2-9f7c-de47ebd045ee" (UID: "60d2b755-b6a5-4ea2-9f7c-de47ebd045ee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:16:06.675009 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:06.674985 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-kube-api-access-8gl7r" (OuterVolumeSpecName: "kube-api-access-8gl7r") pod "60d2b755-b6a5-4ea2-9f7c-de47ebd045ee" (UID: "60d2b755-b6a5-4ea2-9f7c-de47ebd045ee"). InnerVolumeSpecName "kube-api-access-8gl7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:16:06.675112 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:06.675007 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "60d2b755-b6a5-4ea2-9f7c-de47ebd045ee" (UID: "60d2b755-b6a5-4ea2-9f7c-de47ebd045ee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:16:06.774053 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:06.774016 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gl7r\" (UniqueName: \"kubernetes.io/projected/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-kube-api-access-8gl7r\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:16:06.774053 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:06.774046 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-openshift-service-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:16:06.774053 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:06.774056 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee-maas-api-tls\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:16:07.526727 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:07.526699 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" Jul 18 02:16:07.526727 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:07.526718 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd" event={"ID":"60d2b755-b6a5-4ea2-9f7c-de47ebd045ee","Type":"ContainerDied","Data":"6dda427d6850ee0d02ca508e30f2249b1ffbb12974bf58f956cd805688093949"} Jul 18 02:16:07.527250 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:07.526773 2570 scope.go:117] "RemoveContainer" containerID="345a5a84f7ce2f36f8c6c55a9124fee2ab45bd5cc4264b4654f49380cf86c913" Jul 18 02:16:07.541948 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:07.541922 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd"] Jul 18 02:16:07.545886 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:07.545862 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d45c47d8-d588bf465-zvcgd"] Jul 18 02:16:08.894882 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:16:08.894851 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60d2b755-b6a5-4ea2-9f7c-de47ebd045ee" path="/var/lib/kubelet/pods/60d2b755-b6a5-4ea2-9f7c-de47ebd045ee/volumes" Jul 18 02:17:40.956965 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:17:40.956896 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 02:17:40.962749 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:17:40.962725 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 02:17:40.970365 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:17:40.970344 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 02:17:40.976221 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:17:40.976200 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 02:20:16.042928 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.042889 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz"] Jul 18 02:20:16.045424 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.043467 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b4ab932-1d25-467c-982b-6d915da5c228" containerName="istio-proxy" Jul 18 02:20:16.045424 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.043487 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b4ab932-1d25-467c-982b-6d915da5c228" containerName="istio-proxy" Jul 18 02:20:16.045424 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.043518 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60d2b755-b6a5-4ea2-9f7c-de47ebd045ee" containerName="maas-api" Jul 18 02:20:16.045424 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.043527 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="60d2b755-b6a5-4ea2-9f7c-de47ebd045ee" containerName="maas-api" Jul 18 02:20:16.045424 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.043663 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="60d2b755-b6a5-4ea2-9f7c-de47ebd045ee" containerName="maas-api" Jul 18 02:20:16.046887 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.046871 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" Jul 18 02:20:16.049166 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.049142 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e9b53c5b-serving-cert\"" Jul 18 02:20:16.059710 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.059689 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz"] Jul 18 02:20:16.154440 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.154397 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d80f6d2b-6483-4202-a533-557e5aa15f5d-openshift-service-ca\") pod \"maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz\" (UID: \"d80f6d2b-6483-4202-a533-557e5aa15f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" Jul 18 02:20:16.154677 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.154515 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d80f6d2b-6483-4202-a533-557e5aa15f5d-maas-api-tls\") pod \"maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz\" (UID: \"d80f6d2b-6483-4202-a533-557e5aa15f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" Jul 18 02:20:16.154677 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.154549 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt42s\" (UniqueName: \"kubernetes.io/projected/d80f6d2b-6483-4202-a533-557e5aa15f5d-kube-api-access-bt42s\") pod \"maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz\" (UID: \"d80f6d2b-6483-4202-a533-557e5aa15f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" Jul 18 02:20:16.255862 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.255824 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt42s\" (UniqueName: \"kubernetes.io/projected/d80f6d2b-6483-4202-a533-557e5aa15f5d-kube-api-access-bt42s\") pod \"maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz\" (UID: \"d80f6d2b-6483-4202-a533-557e5aa15f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" Jul 18 02:20:16.256064 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.255899 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d80f6d2b-6483-4202-a533-557e5aa15f5d-openshift-service-ca\") pod \"maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz\" (UID: \"d80f6d2b-6483-4202-a533-557e5aa15f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" Jul 18 02:20:16.256064 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.255981 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d80f6d2b-6483-4202-a533-557e5aa15f5d-maas-api-tls\") pod \"maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz\" (UID: \"d80f6d2b-6483-4202-a533-557e5aa15f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" Jul 18 02:20:16.256615 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.256585 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d80f6d2b-6483-4202-a533-557e5aa15f5d-openshift-service-ca\") pod \"maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz\" (UID: \"d80f6d2b-6483-4202-a533-557e5aa15f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" Jul 18 02:20:16.258367 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.258340 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d80f6d2b-6483-4202-a533-557e5aa15f5d-maas-api-tls\") pod \"maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz\" (UID: \"d80f6d2b-6483-4202-a533-557e5aa15f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" Jul 18 02:20:16.264494 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.264470 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt42s\" (UniqueName: \"kubernetes.io/projected/d80f6d2b-6483-4202-a533-557e5aa15f5d-kube-api-access-bt42s\") pod \"maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz\" (UID: \"d80f6d2b-6483-4202-a533-557e5aa15f5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" Jul 18 02:20:16.359678 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.359627 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" Jul 18 02:20:16.701333 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.701302 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz"] Jul 18 02:20:16.704799 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:20:16.704765 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd80f6d2b_6483_4202_a533_557e5aa15f5d.slice/crio-57c09815c6a2ac0c9275df7d3c9edf6140ce94d94970fcd8ba013a117787bd87 WatchSource:0}: Error finding container 57c09815c6a2ac0c9275df7d3c9edf6140ce94d94970fcd8ba013a117787bd87: Status 404 returned error can't find the container with id 57c09815c6a2ac0c9275df7d3c9edf6140ce94d94970fcd8ba013a117787bd87 Jul 18 02:20:16.797098 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:16.797063 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" event={"ID":"d80f6d2b-6483-4202-a533-557e5aa15f5d","Type":"ContainerStarted","Data":"57c09815c6a2ac0c9275df7d3c9edf6140ce94d94970fcd8ba013a117787bd87"} Jul 18 02:20:17.804133 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:17.804097 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" event={"ID":"d80f6d2b-6483-4202-a533-557e5aa15f5d","Type":"ContainerStarted","Data":"735bb36c888d012a4004c1ab0b26bba01a3706c4e993b5cc6e125fe5e2b5c1dc"} Jul 18 02:20:17.804514 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:17.804161 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" Jul 18 02:20:17.820559 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:17.820505 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" podStartSLOduration=1.474427838 podStartE2EDuration="1.82049022s" totalImagesPullingTime="346.062382ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:20:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:20:17.819026247 +0000 UTC m=+1957.496450350" watchObservedRunningTime="2026-07-18 02:20:17.82049022 +0000 UTC m=+1957.497914368" Jul 18 02:20:23.815035 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:20:23.815006 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" Jul 18 02:21:04.068745 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:04.068665 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz"] Jul 18 02:21:04.069298 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:04.068915 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" podUID="d80f6d2b-6483-4202-a533-557e5aa15f5d" containerName="maas-api" containerID="cri-o://735bb36c888d012a4004c1ab0b26bba01a3706c4e993b5cc6e125fe5e2b5c1dc" gracePeriod=30 Jul 18 02:21:04.322132 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:04.322072 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" Jul 18 02:21:04.495566 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:04.495529 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d80f6d2b-6483-4202-a533-557e5aa15f5d-maas-api-tls\") pod \"d80f6d2b-6483-4202-a533-557e5aa15f5d\" (UID: \"d80f6d2b-6483-4202-a533-557e5aa15f5d\") " Jul 18 02:21:04.495788 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:04.495703 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d80f6d2b-6483-4202-a533-557e5aa15f5d-openshift-service-ca\") pod \"d80f6d2b-6483-4202-a533-557e5aa15f5d\" (UID: \"d80f6d2b-6483-4202-a533-557e5aa15f5d\") " Jul 18 02:21:04.495788 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:04.495737 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bt42s\" (UniqueName: \"kubernetes.io/projected/d80f6d2b-6483-4202-a533-557e5aa15f5d-kube-api-access-bt42s\") pod \"d80f6d2b-6483-4202-a533-557e5aa15f5d\" (UID: \"d80f6d2b-6483-4202-a533-557e5aa15f5d\") " Jul 18 02:21:04.496042 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:04.496019 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d80f6d2b-6483-4202-a533-557e5aa15f5d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d80f6d2b-6483-4202-a533-557e5aa15f5d" (UID: "d80f6d2b-6483-4202-a533-557e5aa15f5d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:21:04.497607 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:04.497586 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d80f6d2b-6483-4202-a533-557e5aa15f5d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d80f6d2b-6483-4202-a533-557e5aa15f5d" (UID: "d80f6d2b-6483-4202-a533-557e5aa15f5d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:21:04.497716 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:04.497697 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d80f6d2b-6483-4202-a533-557e5aa15f5d-kube-api-access-bt42s" (OuterVolumeSpecName: "kube-api-access-bt42s") pod "d80f6d2b-6483-4202-a533-557e5aa15f5d" (UID: "d80f6d2b-6483-4202-a533-557e5aa15f5d"). InnerVolumeSpecName "kube-api-access-bt42s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:21:04.596700 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:04.596581 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d80f6d2b-6483-4202-a533-557e5aa15f5d-openshift-service-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:21:04.596700 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:04.596610 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bt42s\" (UniqueName: \"kubernetes.io/projected/d80f6d2b-6483-4202-a533-557e5aa15f5d-kube-api-access-bt42s\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:21:04.596700 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:04.596620 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d80f6d2b-6483-4202-a533-557e5aa15f5d-maas-api-tls\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:21:05.039235 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:05.039196 2570 generic.go:356] "Generic (PLEG): container finished" podID="d80f6d2b-6483-4202-a533-557e5aa15f5d" containerID="735bb36c888d012a4004c1ab0b26bba01a3706c4e993b5cc6e125fe5e2b5c1dc" exitCode=0 Jul 18 02:21:05.039417 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:05.039249 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" Jul 18 02:21:05.039417 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:05.039269 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" event={"ID":"d80f6d2b-6483-4202-a533-557e5aa15f5d","Type":"ContainerDied","Data":"735bb36c888d012a4004c1ab0b26bba01a3706c4e993b5cc6e125fe5e2b5c1dc"} Jul 18 02:21:05.039417 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:05.039321 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz" event={"ID":"d80f6d2b-6483-4202-a533-557e5aa15f5d","Type":"ContainerDied","Data":"57c09815c6a2ac0c9275df7d3c9edf6140ce94d94970fcd8ba013a117787bd87"} Jul 18 02:21:05.039417 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:05.039349 2570 scope.go:117] "RemoveContainer" containerID="735bb36c888d012a4004c1ab0b26bba01a3706c4e993b5cc6e125fe5e2b5c1dc" Jul 18 02:21:05.048935 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:05.048916 2570 scope.go:117] "RemoveContainer" containerID="735bb36c888d012a4004c1ab0b26bba01a3706c4e993b5cc6e125fe5e2b5c1dc" Jul 18 02:21:05.049214 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:21:05.049193 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"735bb36c888d012a4004c1ab0b26bba01a3706c4e993b5cc6e125fe5e2b5c1dc\": container with ID starting with 735bb36c888d012a4004c1ab0b26bba01a3706c4e993b5cc6e125fe5e2b5c1dc not found: ID does not exist" containerID="735bb36c888d012a4004c1ab0b26bba01a3706c4e993b5cc6e125fe5e2b5c1dc" Jul 18 02:21:05.049294 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:05.049222 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"735bb36c888d012a4004c1ab0b26bba01a3706c4e993b5cc6e125fe5e2b5c1dc"} err="failed to get container status \"735bb36c888d012a4004c1ab0b26bba01a3706c4e993b5cc6e125fe5e2b5c1dc\": rpc error: code = NotFound desc = could not find container \"735bb36c888d012a4004c1ab0b26bba01a3706c4e993b5cc6e125fe5e2b5c1dc\": container with ID starting with 735bb36c888d012a4004c1ab0b26bba01a3706c4e993b5cc6e125fe5e2b5c1dc not found: ID does not exist" Jul 18 02:21:05.053815 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:05.053788 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz"] Jul 18 02:21:05.057694 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:05.057671 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9b53c5b-9fcd9bc58-j46sz"] Jul 18 02:21:06.894427 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:21:06.894393 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d80f6d2b-6483-4202-a533-557e5aa15f5d" path="/var/lib/kubelet/pods/d80f6d2b-6483-4202-a533-557e5aa15f5d/volumes" Jul 18 02:22:40.988767 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:40.988738 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 02:22:40.994661 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:40.994623 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 02:22:41.000021 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:41.000001 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 02:22:41.005859 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:41.005841 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 02:22:58.015361 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.015322 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq"] Jul 18 02:22:58.015796 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.015779 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d80f6d2b-6483-4202-a533-557e5aa15f5d" containerName="maas-api" Jul 18 02:22:58.015796 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.015792 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d80f6d2b-6483-4202-a533-557e5aa15f5d" containerName="maas-api" Jul 18 02:22:58.015870 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.015866 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d80f6d2b-6483-4202-a533-557e5aa15f5d" containerName="maas-api" Jul 18 02:22:58.019087 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.019071 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" Jul 18 02:22:58.021206 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.021186 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-289d75-serving-cert\"" Jul 18 02:22:58.028105 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.028081 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq"] Jul 18 02:22:58.132399 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.132360 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hjpg\" (UniqueName: \"kubernetes.io/projected/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-kube-api-access-6hjpg\") pod \"maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq\" (UID: \"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" Jul 18 02:22:58.132590 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.132490 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq\" (UID: \"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" Jul 18 02:22:58.132590 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.132512 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-maas-api-tls\") pod \"maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq\" (UID: \"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" Jul 18 02:22:58.233847 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.233807 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq\" (UID: \"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" Jul 18 02:22:58.234015 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.233852 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-maas-api-tls\") pod \"maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq\" (UID: \"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" Jul 18 02:22:58.234015 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.233904 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hjpg\" (UniqueName: \"kubernetes.io/projected/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-kube-api-access-6hjpg\") pod \"maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq\" (UID: \"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" Jul 18 02:22:58.234114 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:22:58.234019 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-serving-cert: secret "maas-api-e2e-inf-a-289d75-serving-cert" not found Jul 18 02:22:58.234114 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:22:58.234098 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-maas-api-tls podName:eaff19c2-a8f2-41e0-af24-4f6fe2908b0d nodeName:}" failed. No retries permitted until 2026-07-18 02:22:58.734076759 +0000 UTC m=+2118.411500850 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-maas-api-tls") pod "maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" (UID: "eaff19c2-a8f2-41e0-af24-4f6fe2908b0d") : secret "maas-api-e2e-inf-a-289d75-serving-cert" not found Jul 18 02:22:58.234612 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.234591 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-openshift-service-ca\") pod \"maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq\" (UID: \"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" Jul 18 02:22:58.243422 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.243393 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hjpg\" (UniqueName: \"kubernetes.io/projected/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-kube-api-access-6hjpg\") pod \"maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq\" (UID: \"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" Jul 18 02:22:58.738253 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.738214 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-maas-api-tls\") pod \"maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq\" (UID: \"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" Jul 18 02:22:58.740687 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.740660 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-maas-api-tls\") pod \"maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq\" (UID: \"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" Jul 18 02:22:58.931602 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:58.931568 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" Jul 18 02:22:59.088848 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:59.088824 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq"] Jul 18 02:22:59.090841 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:22:59.090815 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaff19c2_a8f2_41e0_af24_4f6fe2908b0d.slice/crio-be49dbcca6e9f05ec767437a078a1d570b81df045ca27edfce3de4bd7e760ba3 WatchSource:0}: Error finding container be49dbcca6e9f05ec767437a078a1d570b81df045ca27edfce3de4bd7e760ba3: Status 404 returned error can't find the container with id be49dbcca6e9f05ec767437a078a1d570b81df045ca27edfce3de4bd7e760ba3 Jul 18 02:22:59.092317 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:59.092294 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 02:22:59.621097 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:22:59.621059 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" event={"ID":"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d","Type":"ContainerStarted","Data":"be49dbcca6e9f05ec767437a078a1d570b81df045ca27edfce3de4bd7e760ba3"} Jul 18 02:23:00.628957 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:23:00.628921 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" event={"ID":"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d","Type":"ContainerStarted","Data":"310fccb63933355685060a381f775635c1ccfc397b6dace721bce0d9f47f1fc9"} Jul 18 02:23:00.629409 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:23:00.629004 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" Jul 18 02:23:00.646933 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:23:00.646884 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" podStartSLOduration=2.838625755 podStartE2EDuration="3.64686716s" totalImagesPullingTime="808.241405ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:22:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:23:00.645208125 +0000 UTC m=+2120.322632236" watchObservedRunningTime="2026-07-18 02:23:00.64686716 +0000 UTC m=+2120.324291271" Jul 18 02:23:06.639683 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:23:06.639635 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" Jul 18 02:24:20.271970 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:20.271932 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq"] Jul 18 02:24:20.273010 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:20.272192 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" podUID="eaff19c2-a8f2-41e0-af24-4f6fe2908b0d" containerName="maas-api" containerID="cri-o://310fccb63933355685060a381f775635c1ccfc397b6dace721bce0d9f47f1fc9" gracePeriod=30 Jul 18 02:24:20.729933 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:20.729907 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" Jul 18 02:24:20.799171 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:20.799142 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hjpg\" (UniqueName: \"kubernetes.io/projected/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-kube-api-access-6hjpg\") pod \"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d\" (UID: \"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d\") " Jul 18 02:24:20.799334 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:20.799185 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-maas-api-tls\") pod \"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d\" (UID: \"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d\") " Jul 18 02:24:20.799334 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:20.799243 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-openshift-service-ca\") pod \"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d\" (UID: \"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d\") " Jul 18 02:24:20.799634 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:20.799609 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "eaff19c2-a8f2-41e0-af24-4f6fe2908b0d" (UID: "eaff19c2-a8f2-41e0-af24-4f6fe2908b0d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:24:20.801435 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:20.801411 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "eaff19c2-a8f2-41e0-af24-4f6fe2908b0d" (UID: "eaff19c2-a8f2-41e0-af24-4f6fe2908b0d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:24:20.801521 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:20.801411 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-kube-api-access-6hjpg" (OuterVolumeSpecName: "kube-api-access-6hjpg") pod "eaff19c2-a8f2-41e0-af24-4f6fe2908b0d" (UID: "eaff19c2-a8f2-41e0-af24-4f6fe2908b0d"). InnerVolumeSpecName "kube-api-access-6hjpg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:24:20.899956 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:20.899927 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hjpg\" (UniqueName: \"kubernetes.io/projected/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-kube-api-access-6hjpg\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:24:20.899956 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:20.899952 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-maas-api-tls\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:24:20.899956 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:20.899961 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d-openshift-service-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:24:21.043286 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:21.043254 2570 generic.go:356] "Generic (PLEG): container finished" podID="eaff19c2-a8f2-41e0-af24-4f6fe2908b0d" containerID="310fccb63933355685060a381f775635c1ccfc397b6dace721bce0d9f47f1fc9" exitCode=0 Jul 18 02:24:21.043474 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:21.043300 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" Jul 18 02:24:21.043474 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:21.043344 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" event={"ID":"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d","Type":"ContainerDied","Data":"310fccb63933355685060a381f775635c1ccfc397b6dace721bce0d9f47f1fc9"} Jul 18 02:24:21.043474 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:21.043393 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq" event={"ID":"eaff19c2-a8f2-41e0-af24-4f6fe2908b0d","Type":"ContainerDied","Data":"be49dbcca6e9f05ec767437a078a1d570b81df045ca27edfce3de4bd7e760ba3"} Jul 18 02:24:21.043474 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:21.043414 2570 scope.go:117] "RemoveContainer" containerID="310fccb63933355685060a381f775635c1ccfc397b6dace721bce0d9f47f1fc9" Jul 18 02:24:21.052296 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:21.052275 2570 scope.go:117] "RemoveContainer" containerID="310fccb63933355685060a381f775635c1ccfc397b6dace721bce0d9f47f1fc9" Jul 18 02:24:21.052611 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:24:21.052590 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"310fccb63933355685060a381f775635c1ccfc397b6dace721bce0d9f47f1fc9\": container with ID starting with 310fccb63933355685060a381f775635c1ccfc397b6dace721bce0d9f47f1fc9 not found: ID does not exist" containerID="310fccb63933355685060a381f775635c1ccfc397b6dace721bce0d9f47f1fc9" Jul 18 02:24:21.052684 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:21.052621 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"310fccb63933355685060a381f775635c1ccfc397b6dace721bce0d9f47f1fc9"} err="failed to get container status \"310fccb63933355685060a381f775635c1ccfc397b6dace721bce0d9f47f1fc9\": rpc error: code = NotFound desc = could not find container \"310fccb63933355685060a381f775635c1ccfc397b6dace721bce0d9f47f1fc9\": container with ID starting with 310fccb63933355685060a381f775635c1ccfc397b6dace721bce0d9f47f1fc9 not found: ID does not exist" Jul 18 02:24:21.061678 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:21.061652 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq"] Jul 18 02:24:21.065012 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:21.064992 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-289d75-784dd4bd47-kmnhq"] Jul 18 02:24:22.895518 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:24:22.895477 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eaff19c2-a8f2-41e0-af24-4f6fe2908b0d" path="/var/lib/kubelet/pods/eaff19c2-a8f2-41e0-af24-4f6fe2908b0d/volumes" Jul 18 02:25:05.713382 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:05.713303 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x"] Jul 18 02:25:05.713853 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:05.713780 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eaff19c2-a8f2-41e0-af24-4f6fe2908b0d" containerName="maas-api" Jul 18 02:25:05.713853 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:05.713794 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="eaff19c2-a8f2-41e0-af24-4f6fe2908b0d" containerName="maas-api" Jul 18 02:25:05.713937 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:05.713880 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="eaff19c2-a8f2-41e0-af24-4f6fe2908b0d" containerName="maas-api" Jul 18 02:25:05.717139 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:05.717110 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" Jul 18 02:25:05.719924 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:05.719903 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-21cdac-serving-cert\"" Jul 18 02:25:05.733564 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:05.733531 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x"] Jul 18 02:25:05.812174 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:05.812122 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b4d9cbe-bf06-45a4-936c-9651e82d7862-openshift-service-ca\") pod \"maas-api-e2e-api-b-21cdac-66f498975b-l496x\" (UID: \"8b4d9cbe-bf06-45a4-936c-9651e82d7862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" Jul 18 02:25:05.812174 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:05.812169 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n2k4\" (UniqueName: \"kubernetes.io/projected/8b4d9cbe-bf06-45a4-936c-9651e82d7862-kube-api-access-2n2k4\") pod \"maas-api-e2e-api-b-21cdac-66f498975b-l496x\" (UID: \"8b4d9cbe-bf06-45a4-936c-9651e82d7862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" Jul 18 02:25:05.812507 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:05.812245 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4d9cbe-bf06-45a4-936c-9651e82d7862-maas-api-tls\") pod \"maas-api-e2e-api-b-21cdac-66f498975b-l496x\" (UID: \"8b4d9cbe-bf06-45a4-936c-9651e82d7862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" Jul 18 02:25:05.913721 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:05.913677 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b4d9cbe-bf06-45a4-936c-9651e82d7862-openshift-service-ca\") pod \"maas-api-e2e-api-b-21cdac-66f498975b-l496x\" (UID: \"8b4d9cbe-bf06-45a4-936c-9651e82d7862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" Jul 18 02:25:05.913721 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:05.913725 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n2k4\" (UniqueName: \"kubernetes.io/projected/8b4d9cbe-bf06-45a4-936c-9651e82d7862-kube-api-access-2n2k4\") pod \"maas-api-e2e-api-b-21cdac-66f498975b-l496x\" (UID: \"8b4d9cbe-bf06-45a4-936c-9651e82d7862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" Jul 18 02:25:05.914005 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:05.913882 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4d9cbe-bf06-45a4-936c-9651e82d7862-maas-api-tls\") pod \"maas-api-e2e-api-b-21cdac-66f498975b-l496x\" (UID: \"8b4d9cbe-bf06-45a4-936c-9651e82d7862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" Jul 18 02:25:05.914091 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:25:05.913996 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-serving-cert: secret "maas-api-e2e-api-b-21cdac-serving-cert" not found Jul 18 02:25:05.914091 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:25:05.914090 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4d9cbe-bf06-45a4-936c-9651e82d7862-maas-api-tls podName:8b4d9cbe-bf06-45a4-936c-9651e82d7862 nodeName:}" failed. No retries permitted until 2026-07-18 02:25:06.414069507 +0000 UTC m=+2246.091493595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8b4d9cbe-bf06-45a4-936c-9651e82d7862-maas-api-tls") pod "maas-api-e2e-api-b-21cdac-66f498975b-l496x" (UID: "8b4d9cbe-bf06-45a4-936c-9651e82d7862") : secret "maas-api-e2e-api-b-21cdac-serving-cert" not found Jul 18 02:25:05.914434 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:05.914413 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b4d9cbe-bf06-45a4-936c-9651e82d7862-openshift-service-ca\") pod \"maas-api-e2e-api-b-21cdac-66f498975b-l496x\" (UID: \"8b4d9cbe-bf06-45a4-936c-9651e82d7862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" Jul 18 02:25:05.923553 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:05.923516 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n2k4\" (UniqueName: \"kubernetes.io/projected/8b4d9cbe-bf06-45a4-936c-9651e82d7862-kube-api-access-2n2k4\") pod \"maas-api-e2e-api-b-21cdac-66f498975b-l496x\" (UID: \"8b4d9cbe-bf06-45a4-936c-9651e82d7862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" Jul 18 02:25:06.418276 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:06.418240 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4d9cbe-bf06-45a4-936c-9651e82d7862-maas-api-tls\") pod \"maas-api-e2e-api-b-21cdac-66f498975b-l496x\" (UID: \"8b4d9cbe-bf06-45a4-936c-9651e82d7862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" Jul 18 02:25:06.420801 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:06.420775 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4d9cbe-bf06-45a4-936c-9651e82d7862-maas-api-tls\") pod \"maas-api-e2e-api-b-21cdac-66f498975b-l496x\" (UID: \"8b4d9cbe-bf06-45a4-936c-9651e82d7862\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" Jul 18 02:25:06.629574 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:06.629537 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" Jul 18 02:25:06.782026 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:06.781982 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x"] Jul 18 02:25:06.783461 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:25:06.783437 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b4d9cbe_bf06_45a4_936c_9651e82d7862.slice/crio-d6f016a4f8cffeaeeb4ec24fa52f6b3f768d37e48ad462fb208f27759750d198 WatchSource:0}: Error finding container d6f016a4f8cffeaeeb4ec24fa52f6b3f768d37e48ad462fb208f27759750d198: Status 404 returned error can't find the container with id d6f016a4f8cffeaeeb4ec24fa52f6b3f768d37e48ad462fb208f27759750d198 Jul 18 02:25:07.279531 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:07.279483 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" event={"ID":"8b4d9cbe-bf06-45a4-936c-9651e82d7862","Type":"ContainerStarted","Data":"b24c7d5cd235ef3eef52dc4c424dad67478110aa9456bada474fc175e9ced819"} Jul 18 02:25:07.279750 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:07.279542 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" event={"ID":"8b4d9cbe-bf06-45a4-936c-9651e82d7862","Type":"ContainerStarted","Data":"d6f016a4f8cffeaeeb4ec24fa52f6b3f768d37e48ad462fb208f27759750d198"} Jul 18 02:25:07.279750 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:07.279580 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" Jul 18 02:25:07.312622 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:07.312568 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" podStartSLOduration=1.934785113 podStartE2EDuration="2.312547854s" totalImagesPullingTime="377.762741ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:25:07.311166299 +0000 UTC m=+2246.988590448" watchObservedRunningTime="2026-07-18 02:25:07.312547854 +0000 UTC m=+2246.989972005" Jul 18 02:25:13.290993 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:25:13.290961 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" Jul 18 02:26:00.113489 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.113457 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x"] Jul 18 02:26:00.113978 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.113671 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" podUID="8b4d9cbe-bf06-45a4-936c-9651e82d7862" containerName="maas-api" containerID="cri-o://b24c7d5cd235ef3eef52dc4c424dad67478110aa9456bada474fc175e9ced819" gracePeriod=30 Jul 18 02:26:00.373320 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.373257 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" Jul 18 02:26:00.515541 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.515504 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2n2k4\" (UniqueName: \"kubernetes.io/projected/8b4d9cbe-bf06-45a4-936c-9651e82d7862-kube-api-access-2n2k4\") pod \"8b4d9cbe-bf06-45a4-936c-9651e82d7862\" (UID: \"8b4d9cbe-bf06-45a4-936c-9651e82d7862\") " Jul 18 02:26:00.515771 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.515603 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4d9cbe-bf06-45a4-936c-9651e82d7862-maas-api-tls\") pod \"8b4d9cbe-bf06-45a4-936c-9651e82d7862\" (UID: \"8b4d9cbe-bf06-45a4-936c-9651e82d7862\") " Jul 18 02:26:00.515771 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.515673 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b4d9cbe-bf06-45a4-936c-9651e82d7862-openshift-service-ca\") pod \"8b4d9cbe-bf06-45a4-936c-9651e82d7862\" (UID: \"8b4d9cbe-bf06-45a4-936c-9651e82d7862\") " Jul 18 02:26:00.515995 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.515968 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b4d9cbe-bf06-45a4-936c-9651e82d7862-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8b4d9cbe-bf06-45a4-936c-9651e82d7862" (UID: "8b4d9cbe-bf06-45a4-936c-9651e82d7862"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:26:00.517832 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.517808 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b4d9cbe-bf06-45a4-936c-9651e82d7862-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8b4d9cbe-bf06-45a4-936c-9651e82d7862" (UID: "8b4d9cbe-bf06-45a4-936c-9651e82d7862"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:26:00.517919 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.517815 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b4d9cbe-bf06-45a4-936c-9651e82d7862-kube-api-access-2n2k4" (OuterVolumeSpecName: "kube-api-access-2n2k4") pod "8b4d9cbe-bf06-45a4-936c-9651e82d7862" (UID: "8b4d9cbe-bf06-45a4-936c-9651e82d7862"). InnerVolumeSpecName "kube-api-access-2n2k4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:26:00.559770 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.559739 2570 generic.go:356] "Generic (PLEG): container finished" podID="8b4d9cbe-bf06-45a4-936c-9651e82d7862" containerID="b24c7d5cd235ef3eef52dc4c424dad67478110aa9456bada474fc175e9ced819" exitCode=0 Jul 18 02:26:00.559980 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.559794 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" Jul 18 02:26:00.559980 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.559806 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" event={"ID":"8b4d9cbe-bf06-45a4-936c-9651e82d7862","Type":"ContainerDied","Data":"b24c7d5cd235ef3eef52dc4c424dad67478110aa9456bada474fc175e9ced819"} Jul 18 02:26:00.559980 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.559869 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x" event={"ID":"8b4d9cbe-bf06-45a4-936c-9651e82d7862","Type":"ContainerDied","Data":"d6f016a4f8cffeaeeb4ec24fa52f6b3f768d37e48ad462fb208f27759750d198"} Jul 18 02:26:00.559980 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.559899 2570 scope.go:117] "RemoveContainer" containerID="b24c7d5cd235ef3eef52dc4c424dad67478110aa9456bada474fc175e9ced819" Jul 18 02:26:00.569314 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.569296 2570 scope.go:117] "RemoveContainer" containerID="b24c7d5cd235ef3eef52dc4c424dad67478110aa9456bada474fc175e9ced819" Jul 18 02:26:00.569591 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:26:00.569567 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b24c7d5cd235ef3eef52dc4c424dad67478110aa9456bada474fc175e9ced819\": container with ID starting with b24c7d5cd235ef3eef52dc4c424dad67478110aa9456bada474fc175e9ced819 not found: ID does not exist" containerID="b24c7d5cd235ef3eef52dc4c424dad67478110aa9456bada474fc175e9ced819" Jul 18 02:26:00.569720 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.569603 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b24c7d5cd235ef3eef52dc4c424dad67478110aa9456bada474fc175e9ced819"} err="failed to get container status \"b24c7d5cd235ef3eef52dc4c424dad67478110aa9456bada474fc175e9ced819\": rpc error: code = NotFound desc = could not find container \"b24c7d5cd235ef3eef52dc4c424dad67478110aa9456bada474fc175e9ced819\": container with ID starting with b24c7d5cd235ef3eef52dc4c424dad67478110aa9456bada474fc175e9ced819 not found: ID does not exist" Jul 18 02:26:00.583519 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.583498 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x"] Jul 18 02:26:00.586722 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.586700 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-21cdac-66f498975b-l496x"] Jul 18 02:26:00.617006 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.616971 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2n2k4\" (UniqueName: \"kubernetes.io/projected/8b4d9cbe-bf06-45a4-936c-9651e82d7862-kube-api-access-2n2k4\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:26:00.617006 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.617004 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4d9cbe-bf06-45a4-936c-9651e82d7862-maas-api-tls\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:26:00.617195 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.617018 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b4d9cbe-bf06-45a4-936c-9651e82d7862-openshift-service-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:26:00.895659 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:00.895614 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b4d9cbe-bf06-45a4-936c-9651e82d7862" path="/var/lib/kubelet/pods/8b4d9cbe-bf06-45a4-936c-9651e82d7862/volumes" Jul 18 02:26:23.029035 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.028997 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl"] Jul 18 02:26:23.029652 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.029628 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b4d9cbe-bf06-45a4-936c-9651e82d7862" containerName="maas-api" Jul 18 02:26:23.029708 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.029659 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b4d9cbe-bf06-45a4-936c-9651e82d7862" containerName="maas-api" Jul 18 02:26:23.029813 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.029801 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b4d9cbe-bf06-45a4-936c-9651e82d7862" containerName="maas-api" Jul 18 02:26:23.033524 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.033502 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" Jul 18 02:26:23.038052 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.038027 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-16c040-serving-cert\"" Jul 18 02:26:23.048845 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.048818 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl"] Jul 18 02:26:23.120575 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.120518 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-484nb\" (UniqueName: \"kubernetes.io/projected/2555d1a2-0c25-4792-85f4-3b845f2aee5d-kube-api-access-484nb\") pod \"maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl\" (UID: \"2555d1a2-0c25-4792-85f4-3b845f2aee5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" Jul 18 02:26:23.120575 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.120578 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2555d1a2-0c25-4792-85f4-3b845f2aee5d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl\" (UID: \"2555d1a2-0c25-4792-85f4-3b845f2aee5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" Jul 18 02:26:23.120823 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.120685 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2555d1a2-0c25-4792-85f4-3b845f2aee5d-maas-api-tls\") pod \"maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl\" (UID: \"2555d1a2-0c25-4792-85f4-3b845f2aee5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" Jul 18 02:26:23.221955 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.221919 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2555d1a2-0c25-4792-85f4-3b845f2aee5d-maas-api-tls\") pod \"maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl\" (UID: \"2555d1a2-0c25-4792-85f4-3b845f2aee5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" Jul 18 02:26:23.222147 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.222056 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-484nb\" (UniqueName: \"kubernetes.io/projected/2555d1a2-0c25-4792-85f4-3b845f2aee5d-kube-api-access-484nb\") pod \"maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl\" (UID: \"2555d1a2-0c25-4792-85f4-3b845f2aee5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" Jul 18 02:26:23.222147 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:26:23.222083 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-serving-cert: secret "maas-api-e2e-shared-b-16c040-serving-cert" not found Jul 18 02:26:23.222147 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.222095 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2555d1a2-0c25-4792-85f4-3b845f2aee5d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl\" (UID: \"2555d1a2-0c25-4792-85f4-3b845f2aee5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" Jul 18 02:26:23.222325 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:26:23.222173 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2555d1a2-0c25-4792-85f4-3b845f2aee5d-maas-api-tls podName:2555d1a2-0c25-4792-85f4-3b845f2aee5d nodeName:}" failed. No retries permitted until 2026-07-18 02:26:23.722151917 +0000 UTC m=+2323.399576019 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2555d1a2-0c25-4792-85f4-3b845f2aee5d-maas-api-tls") pod "maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" (UID: "2555d1a2-0c25-4792-85f4-3b845f2aee5d") : secret "maas-api-e2e-shared-b-16c040-serving-cert" not found Jul 18 02:26:23.222724 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.222706 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2555d1a2-0c25-4792-85f4-3b845f2aee5d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl\" (UID: \"2555d1a2-0c25-4792-85f4-3b845f2aee5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" Jul 18 02:26:23.232231 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.232195 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-484nb\" (UniqueName: \"kubernetes.io/projected/2555d1a2-0c25-4792-85f4-3b845f2aee5d-kube-api-access-484nb\") pod \"maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl\" (UID: \"2555d1a2-0c25-4792-85f4-3b845f2aee5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" Jul 18 02:26:23.727327 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.727292 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2555d1a2-0c25-4792-85f4-3b845f2aee5d-maas-api-tls\") pod \"maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl\" (UID: \"2555d1a2-0c25-4792-85f4-3b845f2aee5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" Jul 18 02:26:23.729798 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.729772 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2555d1a2-0c25-4792-85f4-3b845f2aee5d-maas-api-tls\") pod \"maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl\" (UID: \"2555d1a2-0c25-4792-85f4-3b845f2aee5d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" Jul 18 02:26:23.946331 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:23.946297 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" Jul 18 02:26:24.092495 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:24.092466 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl"] Jul 18 02:26:24.094739 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:26:24.094711 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2555d1a2_0c25_4792_85f4_3b845f2aee5d.slice/crio-c4a5d2f64daa16ca4930477afe865bcadde146fa76e2f063a7f19d02958145fb WatchSource:0}: Error finding container c4a5d2f64daa16ca4930477afe865bcadde146fa76e2f063a7f19d02958145fb: Status 404 returned error can't find the container with id c4a5d2f64daa16ca4930477afe865bcadde146fa76e2f063a7f19d02958145fb Jul 18 02:26:24.683948 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:24.683841 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" event={"ID":"2555d1a2-0c25-4792-85f4-3b845f2aee5d","Type":"ContainerStarted","Data":"cd88dec291b1702abb8830109fb8b5e6103d27d568913632e8bfdd41813e4b7b"} Jul 18 02:26:24.683948 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:24.683903 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" event={"ID":"2555d1a2-0c25-4792-85f4-3b845f2aee5d","Type":"ContainerStarted","Data":"c4a5d2f64daa16ca4930477afe865bcadde146fa76e2f063a7f19d02958145fb"} Jul 18 02:26:24.683948 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:24.683930 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" Jul 18 02:26:24.700201 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:24.700153 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" podStartSLOduration=2.42496075 podStartE2EDuration="2.700137345s" totalImagesPullingTime="275.176595ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:26:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:26:24.699126851 +0000 UTC m=+2324.376550975" watchObservedRunningTime="2026-07-18 02:26:24.700137345 +0000 UTC m=+2324.377561574" Jul 18 02:26:30.695813 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:26:30.695779 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" Jul 18 02:27:41.030456 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:27:41.030428 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 02:27:41.037285 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:27:41.037258 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 02:27:41.038796 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:27:41.038773 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 02:27:41.044316 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:27:41.044296 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 02:32:41.065061 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:32:41.064971 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 02:32:41.069397 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:32:41.069372 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 02:32:41.070652 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:32:41.070624 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 02:32:41.074909 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:32:41.074888 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jshl6_51675a11-0cb3-4fa9-99db-3cd90c1355fa/ovn-acl-logging/0.log" Jul 18 02:33:23.498199 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:23.498161 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7428-140098145509040"] Jul 18 02:33:23.501714 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:23.501696 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145509040" Jul 18 02:33:23.504149 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:23.504125 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-46gsq\"" Jul 18 02:33:23.507150 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:23.507114 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-140098145509040"] Jul 18 02:33:23.694283 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:23.694244 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtxt5\" (UniqueName: \"kubernetes.io/projected/9f81dd2e-8be1-4539-b5db-96728688c2ad-kube-api-access-rtxt5\") pod \"test-curl-7428-140098145509040\" (UID: \"9f81dd2e-8be1-4539-b5db-96728688c2ad\") " pod="opendatahub/test-curl-7428-140098145509040" Jul 18 02:33:23.795360 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:23.795260 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtxt5\" (UniqueName: \"kubernetes.io/projected/9f81dd2e-8be1-4539-b5db-96728688c2ad-kube-api-access-rtxt5\") pod \"test-curl-7428-140098145509040\" (UID: \"9f81dd2e-8be1-4539-b5db-96728688c2ad\") " pod="opendatahub/test-curl-7428-140098145509040" Jul 18 02:33:23.803309 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:23.803277 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtxt5\" (UniqueName: \"kubernetes.io/projected/9f81dd2e-8be1-4539-b5db-96728688c2ad-kube-api-access-rtxt5\") pod \"test-curl-7428-140098145509040\" (UID: \"9f81dd2e-8be1-4539-b5db-96728688c2ad\") " pod="opendatahub/test-curl-7428-140098145509040" Jul 18 02:33:23.812633 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:23.812606 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145509040" Jul 18 02:33:23.954986 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:23.954960 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-140098145509040"] Jul 18 02:33:23.957154 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:33:23.957126 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f81dd2e_8be1_4539_b5db_96728688c2ad.slice/crio-2a214cdad1462f5dbdecb29987c77f141484f9df50a2bfe75871d0f8cd7d4b73 WatchSource:0}: Error finding container 2a214cdad1462f5dbdecb29987c77f141484f9df50a2bfe75871d0f8cd7d4b73: Status 404 returned error can't find the container with id 2a214cdad1462f5dbdecb29987c77f141484f9df50a2bfe75871d0f8cd7d4b73 Jul 18 02:33:23.958501 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:23.958485 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 02:33:24.858094 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:24.858059 2570 generic.go:356] "Generic (PLEG): container finished" podID="9f81dd2e-8be1-4539-b5db-96728688c2ad" containerID="cb70dab0fdd4c4b55cce9411a7b42cf200113328e35850f778601f6f62a0524a" exitCode=0 Jul 18 02:33:24.858486 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:24.858125 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-140098145509040" event={"ID":"9f81dd2e-8be1-4539-b5db-96728688c2ad","Type":"ContainerDied","Data":"cb70dab0fdd4c4b55cce9411a7b42cf200113328e35850f778601f6f62a0524a"} Jul 18 02:33:24.858486 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:24.858152 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-140098145509040" event={"ID":"9f81dd2e-8be1-4539-b5db-96728688c2ad","Type":"ContainerStarted","Data":"2a214cdad1462f5dbdecb29987c77f141484f9df50a2bfe75871d0f8cd7d4b73"} Jul 18 02:33:26.006443 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.006416 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145509040" Jul 18 02:33:26.013142 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.013116 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtxt5\" (UniqueName: \"kubernetes.io/projected/9f81dd2e-8be1-4539-b5db-96728688c2ad-kube-api-access-rtxt5\") pod \"9f81dd2e-8be1-4539-b5db-96728688c2ad\" (UID: \"9f81dd2e-8be1-4539-b5db-96728688c2ad\") " Jul 18 02:33:26.015100 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.015080 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f81dd2e-8be1-4539-b5db-96728688c2ad-kube-api-access-rtxt5" (OuterVolumeSpecName: "kube-api-access-rtxt5") pod "9f81dd2e-8be1-4539-b5db-96728688c2ad" (UID: "9f81dd2e-8be1-4539-b5db-96728688c2ad"). InnerVolumeSpecName "kube-api-access-rtxt5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:33:26.039475 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.039431 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7428-140098145509040_9f81dd2e-8be1-4539-b5db-96728688c2ad/test-curl-7428-140098145509040/0.log" Jul 18 02:33:26.063157 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.063115 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7428-140098145509040"] Jul 18 02:33:26.067259 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.067234 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7428-140098145509040"] Jul 18 02:33:26.114487 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.114410 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtxt5\" (UniqueName: \"kubernetes.io/projected/9f81dd2e-8be1-4539-b5db-96728688c2ad-kube-api-access-rtxt5\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:33:26.187766 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.187731 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7428-140098145508016"] Jul 18 02:33:26.188979 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.188951 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f81dd2e-8be1-4539-b5db-96728688c2ad" containerName="test-curl-7428-140098145509040" Jul 18 02:33:26.189148 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.189135 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f81dd2e-8be1-4539-b5db-96728688c2ad" containerName="test-curl-7428-140098145509040" Jul 18 02:33:26.189452 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.189435 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f81dd2e-8be1-4539-b5db-96728688c2ad" containerName="test-curl-7428-140098145509040" Jul 18 02:33:26.194305 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.194065 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145508016" Jul 18 02:33:26.197529 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.197501 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-140098145508016"] Jul 18 02:33:26.215012 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.214977 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzbhp\" (UniqueName: \"kubernetes.io/projected/2fd97122-04a0-4434-b4e1-69c91d03ccfe-kube-api-access-tzbhp\") pod \"test-curl-7428-140098145508016\" (UID: \"2fd97122-04a0-4434-b4e1-69c91d03ccfe\") " pod="opendatahub/test-curl-7428-140098145508016" Jul 18 02:33:26.316114 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.316077 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzbhp\" (UniqueName: \"kubernetes.io/projected/2fd97122-04a0-4434-b4e1-69c91d03ccfe-kube-api-access-tzbhp\") pod \"test-curl-7428-140098145508016\" (UID: \"2fd97122-04a0-4434-b4e1-69c91d03ccfe\") " pod="opendatahub/test-curl-7428-140098145508016" Jul 18 02:33:26.324439 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.324417 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzbhp\" (UniqueName: \"kubernetes.io/projected/2fd97122-04a0-4434-b4e1-69c91d03ccfe-kube-api-access-tzbhp\") pod \"test-curl-7428-140098145508016\" (UID: \"2fd97122-04a0-4434-b4e1-69c91d03ccfe\") " pod="opendatahub/test-curl-7428-140098145508016" Jul 18 02:33:26.506967 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.506879 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145508016" Jul 18 02:33:26.646437 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.646412 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-140098145508016"] Jul 18 02:33:26.648785 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:33:26.648755 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fd97122_04a0_4434_b4e1_69c91d03ccfe.slice/crio-8e7a7b13bcda8a286f06017ef4f100e143539d27c31e3152cb36538126291046 WatchSource:0}: Error finding container 8e7a7b13bcda8a286f06017ef4f100e143539d27c31e3152cb36538126291046: Status 404 returned error can't find the container with id 8e7a7b13bcda8a286f06017ef4f100e143539d27c31e3152cb36538126291046 Jul 18 02:33:26.870216 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.870180 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-140098145508016" event={"ID":"2fd97122-04a0-4434-b4e1-69c91d03ccfe","Type":"ContainerStarted","Data":"8e7a7b13bcda8a286f06017ef4f100e143539d27c31e3152cb36538126291046"} Jul 18 02:33:26.871353 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.871332 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2a214cdad1462f5dbdecb29987c77f141484f9df50a2bfe75871d0f8cd7d4b73" Jul 18 02:33:26.871353 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.871332 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145509040" Jul 18 02:33:26.881411 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:33:26.881383 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7428-140098145509040\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="9f81dd2e-8be1-4539-b5db-96728688c2ad" pod="opendatahub/test-curl-7428-140098145509040" Jul 18 02:33:26.894994 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:26.894965 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f81dd2e-8be1-4539-b5db-96728688c2ad" path="/var/lib/kubelet/pods/9f81dd2e-8be1-4539-b5db-96728688c2ad/volumes" Jul 18 02:33:27.877980 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:27.877941 2570 generic.go:356] "Generic (PLEG): container finished" podID="2fd97122-04a0-4434-b4e1-69c91d03ccfe" containerID="db30dd78c75c3c74b06482a15ce0af9d032315f8b4b27ff213f6edd80fcc60b9" exitCode=0 Jul 18 02:33:27.878361 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:27.878028 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-140098145508016" event={"ID":"2fd97122-04a0-4434-b4e1-69c91d03ccfe","Type":"ContainerDied","Data":"db30dd78c75c3c74b06482a15ce0af9d032315f8b4b27ff213f6edd80fcc60b9"} Jul 18 02:33:29.031538 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.031510 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145508016" Jul 18 02:33:29.041093 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.041071 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzbhp\" (UniqueName: \"kubernetes.io/projected/2fd97122-04a0-4434-b4e1-69c91d03ccfe-kube-api-access-tzbhp\") pod \"2fd97122-04a0-4434-b4e1-69c91d03ccfe\" (UID: \"2fd97122-04a0-4434-b4e1-69c91d03ccfe\") " Jul 18 02:33:29.043104 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.043073 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fd97122-04a0-4434-b4e1-69c91d03ccfe-kube-api-access-tzbhp" (OuterVolumeSpecName: "kube-api-access-tzbhp") pod "2fd97122-04a0-4434-b4e1-69c91d03ccfe" (UID: "2fd97122-04a0-4434-b4e1-69c91d03ccfe"). InnerVolumeSpecName "kube-api-access-tzbhp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:33:29.053021 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.052995 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7428-140098145508016_2fd97122-04a0-4434-b4e1-69c91d03ccfe/test-curl-7428-140098145508016/0.log" Jul 18 02:33:29.075229 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.075197 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7428-140098145508016"] Jul 18 02:33:29.078376 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.078344 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7428-140098145508016"] Jul 18 02:33:29.141772 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.141683 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzbhp\" (UniqueName: \"kubernetes.io/projected/2fd97122-04a0-4434-b4e1-69c91d03ccfe-kube-api-access-tzbhp\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:33:29.242398 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.242369 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7428-140098145592208"] Jul 18 02:33:29.242802 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.242789 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fd97122-04a0-4434-b4e1-69c91d03ccfe" containerName="test-curl-7428-140098145508016" Jul 18 02:33:29.242858 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.242804 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fd97122-04a0-4434-b4e1-69c91d03ccfe" containerName="test-curl-7428-140098145508016" Jul 18 02:33:29.242895 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.242876 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fd97122-04a0-4434-b4e1-69c91d03ccfe" containerName="test-curl-7428-140098145508016" Jul 18 02:33:29.245958 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.245942 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145592208" Jul 18 02:33:29.249834 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.249808 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-140098145592208"] Jul 18 02:33:29.343778 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.343733 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm8k8\" (UniqueName: \"kubernetes.io/projected/a19c8ebc-4662-4ca3-9e92-adcb366ab3e5-kube-api-access-mm8k8\") pod \"test-curl-7428-140098145592208\" (UID: \"a19c8ebc-4662-4ca3-9e92-adcb366ab3e5\") " pod="opendatahub/test-curl-7428-140098145592208" Jul 18 02:33:29.444919 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.444822 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm8k8\" (UniqueName: \"kubernetes.io/projected/a19c8ebc-4662-4ca3-9e92-adcb366ab3e5-kube-api-access-mm8k8\") pod \"test-curl-7428-140098145592208\" (UID: \"a19c8ebc-4662-4ca3-9e92-adcb366ab3e5\") " pod="opendatahub/test-curl-7428-140098145592208" Jul 18 02:33:29.452373 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.452343 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm8k8\" (UniqueName: \"kubernetes.io/projected/a19c8ebc-4662-4ca3-9e92-adcb366ab3e5-kube-api-access-mm8k8\") pod \"test-curl-7428-140098145592208\" (UID: \"a19c8ebc-4662-4ca3-9e92-adcb366ab3e5\") " pod="opendatahub/test-curl-7428-140098145592208" Jul 18 02:33:29.556878 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.556812 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145592208" Jul 18 02:33:29.709546 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.709523 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-140098145592208"] Jul 18 02:33:29.711561 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:33:29.711533 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda19c8ebc_4662_4ca3_9e92_adcb366ab3e5.slice/crio-49f130568e08ef72a4dbf09b9dded035b536cadc5a4c8e7b3b68a1a38c6e9d4f WatchSource:0}: Error finding container 49f130568e08ef72a4dbf09b9dded035b536cadc5a4c8e7b3b68a1a38c6e9d4f: Status 404 returned error can't find the container with id 49f130568e08ef72a4dbf09b9dded035b536cadc5a4c8e7b3b68a1a38c6e9d4f Jul 18 02:33:29.889588 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.889558 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e7a7b13bcda8a286f06017ef4f100e143539d27c31e3152cb36538126291046" Jul 18 02:33:29.889588 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.889581 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145508016" Jul 18 02:33:29.890768 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:29.890741 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-140098145592208" event={"ID":"a19c8ebc-4662-4ca3-9e92-adcb366ab3e5","Type":"ContainerStarted","Data":"49f130568e08ef72a4dbf09b9dded035b536cadc5a4c8e7b3b68a1a38c6e9d4f"} Jul 18 02:33:29.900326 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:33:29.900299 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7428-140098145508016\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="2fd97122-04a0-4434-b4e1-69c91d03ccfe" pod="opendatahub/test-curl-7428-140098145508016" Jul 18 02:33:30.893111 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:33:30.893072 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7428-140098145508016\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="2fd97122-04a0-4434-b4e1-69c91d03ccfe" pod="opendatahub/test-curl-7428-140098145508016" Jul 18 02:33:30.895934 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:30.895904 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fd97122-04a0-4434-b4e1-69c91d03ccfe" path="/var/lib/kubelet/pods/2fd97122-04a0-4434-b4e1-69c91d03ccfe/volumes" Jul 18 02:33:30.898021 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:30.897995 2570 generic.go:356] "Generic (PLEG): container finished" podID="a19c8ebc-4662-4ca3-9e92-adcb366ab3e5" containerID="708fccde8e74fb0005b8f32dd737b6cccbb5ac6810a2bec9f3f6d0dd3daed41e" exitCode=0 Jul 18 02:33:30.898158 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:30.898031 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-140098145592208" event={"ID":"a19c8ebc-4662-4ca3-9e92-adcb366ab3e5","Type":"ContainerDied","Data":"708fccde8e74fb0005b8f32dd737b6cccbb5ac6810a2bec9f3f6d0dd3daed41e"} Jul 18 02:33:32.053521 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.053490 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145592208" Jul 18 02:33:32.068503 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.068459 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mm8k8\" (UniqueName: \"kubernetes.io/projected/a19c8ebc-4662-4ca3-9e92-adcb366ab3e5-kube-api-access-mm8k8\") pod \"a19c8ebc-4662-4ca3-9e92-adcb366ab3e5\" (UID: \"a19c8ebc-4662-4ca3-9e92-adcb366ab3e5\") " Jul 18 02:33:32.071192 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.071145 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a19c8ebc-4662-4ca3-9e92-adcb366ab3e5-kube-api-access-mm8k8" (OuterVolumeSpecName: "kube-api-access-mm8k8") pod "a19c8ebc-4662-4ca3-9e92-adcb366ab3e5" (UID: "a19c8ebc-4662-4ca3-9e92-adcb366ab3e5"). InnerVolumeSpecName "kube-api-access-mm8k8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:33:32.073226 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.073199 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7428-140098145592208_a19c8ebc-4662-4ca3-9e92-adcb366ab3e5/test-curl-7428-140098145592208/0.log" Jul 18 02:33:32.101459 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.101423 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7428-140098145592208"] Jul 18 02:33:32.104937 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.104911 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7428-140098145592208"] Jul 18 02:33:32.169438 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.169342 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mm8k8\" (UniqueName: \"kubernetes.io/projected/a19c8ebc-4662-4ca3-9e92-adcb366ab3e5-kube-api-access-mm8k8\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:33:32.223251 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.223218 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7428-140098145592208"] Jul 18 02:33:32.223705 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.223682 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a19c8ebc-4662-4ca3-9e92-adcb366ab3e5" containerName="test-curl-7428-140098145592208" Jul 18 02:33:32.223705 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.223703 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a19c8ebc-4662-4ca3-9e92-adcb366ab3e5" containerName="test-curl-7428-140098145592208" Jul 18 02:33:32.223860 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.223795 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a19c8ebc-4662-4ca3-9e92-adcb366ab3e5" containerName="test-curl-7428-140098145592208" Jul 18 02:33:32.228235 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.227249 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145592208" Jul 18 02:33:32.231396 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.231368 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-140098145592208"] Jul 18 02:33:32.270850 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.270815 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm6w4\" (UniqueName: \"kubernetes.io/projected/f3a6ddc6-8708-43bd-8738-26538ad5da1a-kube-api-access-hm6w4\") pod \"test-curl-7428-140098145592208\" (UID: \"f3a6ddc6-8708-43bd-8738-26538ad5da1a\") " pod="opendatahub/test-curl-7428-140098145592208" Jul 18 02:33:32.371660 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.371609 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm6w4\" (UniqueName: \"kubernetes.io/projected/f3a6ddc6-8708-43bd-8738-26538ad5da1a-kube-api-access-hm6w4\") pod \"test-curl-7428-140098145592208\" (UID: \"f3a6ddc6-8708-43bd-8738-26538ad5da1a\") " pod="opendatahub/test-curl-7428-140098145592208" Jul 18 02:33:32.379552 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.379521 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm6w4\" (UniqueName: \"kubernetes.io/projected/f3a6ddc6-8708-43bd-8738-26538ad5da1a-kube-api-access-hm6w4\") pod \"test-curl-7428-140098145592208\" (UID: \"f3a6ddc6-8708-43bd-8738-26538ad5da1a\") " pod="opendatahub/test-curl-7428-140098145592208" Jul 18 02:33:32.539061 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.538968 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145592208" Jul 18 02:33:32.690968 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.690938 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-140098145592208"] Jul 18 02:33:32.692881 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:33:32.692848 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf3a6ddc6_8708_43bd_8738_26538ad5da1a.slice/crio-26dd1901038ae2d4b0941d038736f80049790f2c5ed36e95a85e29142ff53492 WatchSource:0}: Error finding container 26dd1901038ae2d4b0941d038736f80049790f2c5ed36e95a85e29142ff53492: Status 404 returned error can't find the container with id 26dd1901038ae2d4b0941d038736f80049790f2c5ed36e95a85e29142ff53492 Jul 18 02:33:32.894981 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.894943 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a19c8ebc-4662-4ca3-9e92-adcb366ab3e5" path="/var/lib/kubelet/pods/a19c8ebc-4662-4ca3-9e92-adcb366ab3e5/volumes" Jul 18 02:33:32.910025 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.909995 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145592208" Jul 18 02:33:32.910183 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.910036 2570 scope.go:117] "RemoveContainer" containerID="708fccde8e74fb0005b8f32dd737b6cccbb5ac6810a2bec9f3f6d0dd3daed41e" Jul 18 02:33:32.911361 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:32.911336 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-140098145592208" event={"ID":"f3a6ddc6-8708-43bd-8738-26538ad5da1a","Type":"ContainerStarted","Data":"26dd1901038ae2d4b0941d038736f80049790f2c5ed36e95a85e29142ff53492"} Jul 18 02:33:33.918838 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:33.918804 2570 generic.go:356] "Generic (PLEG): container finished" podID="f3a6ddc6-8708-43bd-8738-26538ad5da1a" containerID="bb60f0975270ef0d7de0404cbb76d6b68beb1103f70bb477c2d714eadc2af00d" exitCode=0 Jul 18 02:33:33.919292 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:33.918888 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-140098145592208" event={"ID":"f3a6ddc6-8708-43bd-8738-26538ad5da1a","Type":"ContainerDied","Data":"bb60f0975270ef0d7de0404cbb76d6b68beb1103f70bb477c2d714eadc2af00d"} Jul 18 02:33:35.073228 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.073199 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145592208" Jul 18 02:33:35.092411 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.092380 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7428-140098145592208_f3a6ddc6-8708-43bd-8738-26538ad5da1a/test-curl-7428-140098145592208/0.log" Jul 18 02:33:35.098470 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.098444 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hm6w4\" (UniqueName: \"kubernetes.io/projected/f3a6ddc6-8708-43bd-8738-26538ad5da1a-kube-api-access-hm6w4\") pod \"f3a6ddc6-8708-43bd-8738-26538ad5da1a\" (UID: \"f3a6ddc6-8708-43bd-8738-26538ad5da1a\") " Jul 18 02:33:35.100565 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.100535 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3a6ddc6-8708-43bd-8738-26538ad5da1a-kube-api-access-hm6w4" (OuterVolumeSpecName: "kube-api-access-hm6w4") pod "f3a6ddc6-8708-43bd-8738-26538ad5da1a" (UID: "f3a6ddc6-8708-43bd-8738-26538ad5da1a"). InnerVolumeSpecName "kube-api-access-hm6w4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:33:35.113781 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.113752 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7428-140098145592208"] Jul 18 02:33:35.117135 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.117113 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7428-140098145592208"] Jul 18 02:33:35.199807 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.199715 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm6w4\" (UniqueName: \"kubernetes.io/projected/f3a6ddc6-8708-43bd-8738-26538ad5da1a-kube-api-access-hm6w4\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:33:35.236840 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.236808 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7428-140098145593216"] Jul 18 02:33:35.237231 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.237220 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3a6ddc6-8708-43bd-8738-26538ad5da1a" containerName="test-curl-7428-140098145592208" Jul 18 02:33:35.237292 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.237233 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3a6ddc6-8708-43bd-8738-26538ad5da1a" containerName="test-curl-7428-140098145592208" Jul 18 02:33:35.237329 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.237305 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3a6ddc6-8708-43bd-8738-26538ad5da1a" containerName="test-curl-7428-140098145592208" Jul 18 02:33:35.240339 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.240322 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145593216" Jul 18 02:33:35.248362 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.248310 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-140098145593216"] Jul 18 02:33:35.300746 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.300713 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcf2k\" (UniqueName: \"kubernetes.io/projected/e3840509-3417-4e2a-a213-ae6b71c7a957-kube-api-access-xcf2k\") pod \"test-curl-7428-140098145593216\" (UID: \"e3840509-3417-4e2a-a213-ae6b71c7a957\") " pod="opendatahub/test-curl-7428-140098145593216" Jul 18 02:33:35.401622 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.401578 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcf2k\" (UniqueName: \"kubernetes.io/projected/e3840509-3417-4e2a-a213-ae6b71c7a957-kube-api-access-xcf2k\") pod \"test-curl-7428-140098145593216\" (UID: \"e3840509-3417-4e2a-a213-ae6b71c7a957\") " pod="opendatahub/test-curl-7428-140098145593216" Jul 18 02:33:35.410334 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.410297 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcf2k\" (UniqueName: \"kubernetes.io/projected/e3840509-3417-4e2a-a213-ae6b71c7a957-kube-api-access-xcf2k\") pod \"test-curl-7428-140098145593216\" (UID: \"e3840509-3417-4e2a-a213-ae6b71c7a957\") " pod="opendatahub/test-curl-7428-140098145593216" Jul 18 02:33:35.551327 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.551294 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145593216" Jul 18 02:33:35.694786 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.694740 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-140098145593216"] Jul 18 02:33:35.697109 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:33:35.697084 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3840509_3417_4e2a_a213_ae6b71c7a957.slice/crio-de5931bbaa86f5c0881e690d8a87b22f5c89f3518738397c2f253a0214d906f6 WatchSource:0}: Error finding container de5931bbaa86f5c0881e690d8a87b22f5c89f3518738397c2f253a0214d906f6: Status 404 returned error can't find the container with id de5931bbaa86f5c0881e690d8a87b22f5c89f3518738397c2f253a0214d906f6 Jul 18 02:33:35.933753 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.933623 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-140098145593216" event={"ID":"e3840509-3417-4e2a-a213-ae6b71c7a957","Type":"ContainerStarted","Data":"de5931bbaa86f5c0881e690d8a87b22f5c89f3518738397c2f253a0214d906f6"} Jul 18 02:33:35.934932 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.934908 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26dd1901038ae2d4b0941d038736f80049790f2c5ed36e95a85e29142ff53492" Jul 18 02:33:35.934932 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:35.934913 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145592208" Jul 18 02:33:35.945430 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:33:35.945400 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7428-140098145592208\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="f3a6ddc6-8708-43bd-8738-26538ad5da1a" pod="opendatahub/test-curl-7428-140098145592208" Jul 18 02:33:36.894901 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:36.894866 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3a6ddc6-8708-43bd-8738-26538ad5da1a" path="/var/lib/kubelet/pods/f3a6ddc6-8708-43bd-8738-26538ad5da1a/volumes" Jul 18 02:33:36.941616 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:36.941575 2570 generic.go:356] "Generic (PLEG): container finished" podID="e3840509-3417-4e2a-a213-ae6b71c7a957" containerID="7666ec405310000b5ed99c2c07a9c10a98d2319d027a344aecaca9f9d29d0a78" exitCode=0 Jul 18 02:33:36.941803 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:36.941629 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-140098145593216" event={"ID":"e3840509-3417-4e2a-a213-ae6b71c7a957","Type":"ContainerDied","Data":"7666ec405310000b5ed99c2c07a9c10a98d2319d027a344aecaca9f9d29d0a78"} Jul 18 02:33:38.113257 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.113230 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145593216" Jul 18 02:33:38.133249 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.133200 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7428-140098145593216_e3840509-3417-4e2a-a213-ae6b71c7a957/test-curl-7428-140098145593216/0.log" Jul 18 02:33:38.157328 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.157291 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7428-140098145593216"] Jul 18 02:33:38.160676 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.160635 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7428-140098145593216"] Jul 18 02:33:38.230741 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.230621 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcf2k\" (UniqueName: \"kubernetes.io/projected/e3840509-3417-4e2a-a213-ae6b71c7a957-kube-api-access-xcf2k\") pod \"e3840509-3417-4e2a-a213-ae6b71c7a957\" (UID: \"e3840509-3417-4e2a-a213-ae6b71c7a957\") " Jul 18 02:33:38.232810 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.232772 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3840509-3417-4e2a-a213-ae6b71c7a957-kube-api-access-xcf2k" (OuterVolumeSpecName: "kube-api-access-xcf2k") pod "e3840509-3417-4e2a-a213-ae6b71c7a957" (UID: "e3840509-3417-4e2a-a213-ae6b71c7a957"). InnerVolumeSpecName "kube-api-access-xcf2k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:33:38.281608 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.281573 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7428-140098145593216"] Jul 18 02:33:38.282009 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.281995 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3840509-3417-4e2a-a213-ae6b71c7a957" containerName="test-curl-7428-140098145593216" Jul 18 02:33:38.282058 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.282010 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3840509-3417-4e2a-a213-ae6b71c7a957" containerName="test-curl-7428-140098145593216" Jul 18 02:33:38.282096 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.282073 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3840509-3417-4e2a-a213-ae6b71c7a957" containerName="test-curl-7428-140098145593216" Jul 18 02:33:38.284935 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.284919 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145593216" Jul 18 02:33:38.292070 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.292039 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-140098145593216"] Jul 18 02:33:38.331558 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.331508 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn7lf\" (UniqueName: \"kubernetes.io/projected/367da045-afa4-48c1-a117-25c808345068-kube-api-access-hn7lf\") pod \"test-curl-7428-140098145593216\" (UID: \"367da045-afa4-48c1-a117-25c808345068\") " pod="opendatahub/test-curl-7428-140098145593216" Jul 18 02:33:38.331776 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.331750 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcf2k\" (UniqueName: \"kubernetes.io/projected/e3840509-3417-4e2a-a213-ae6b71c7a957-kube-api-access-xcf2k\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:33:38.433015 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.432975 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn7lf\" (UniqueName: \"kubernetes.io/projected/367da045-afa4-48c1-a117-25c808345068-kube-api-access-hn7lf\") pod \"test-curl-7428-140098145593216\" (UID: \"367da045-afa4-48c1-a117-25c808345068\") " pod="opendatahub/test-curl-7428-140098145593216" Jul 18 02:33:38.441011 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.440985 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn7lf\" (UniqueName: \"kubernetes.io/projected/367da045-afa4-48c1-a117-25c808345068-kube-api-access-hn7lf\") pod \"test-curl-7428-140098145593216\" (UID: \"367da045-afa4-48c1-a117-25c808345068\") " pod="opendatahub/test-curl-7428-140098145593216" Jul 18 02:33:38.595567 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.595522 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145593216" Jul 18 02:33:38.733311 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.733255 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7428-140098145593216"] Jul 18 02:33:38.735473 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:33:38.735434 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod367da045_afa4_48c1_a117_25c808345068.slice/crio-0ab7591e1134e1b7ee8ad051ddc5d7dda8aefcc447b9ae40eda6db002203b8d9 WatchSource:0}: Error finding container 0ab7591e1134e1b7ee8ad051ddc5d7dda8aefcc447b9ae40eda6db002203b8d9: Status 404 returned error can't find the container with id 0ab7591e1134e1b7ee8ad051ddc5d7dda8aefcc447b9ae40eda6db002203b8d9 Jul 18 02:33:38.896118 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.896029 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3840509-3417-4e2a-a213-ae6b71c7a957" path="/var/lib/kubelet/pods/e3840509-3417-4e2a-a213-ae6b71c7a957/volumes" Jul 18 02:33:38.953579 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.953539 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-140098145593216" event={"ID":"367da045-afa4-48c1-a117-25c808345068","Type":"ContainerStarted","Data":"0ab7591e1134e1b7ee8ad051ddc5d7dda8aefcc447b9ae40eda6db002203b8d9"} Jul 18 02:33:38.954757 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.954736 2570 scope.go:117] "RemoveContainer" containerID="7666ec405310000b5ed99c2c07a9c10a98d2319d027a344aecaca9f9d29d0a78" Jul 18 02:33:38.954881 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:38.954773 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145593216" Jul 18 02:33:39.965592 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:39.965558 2570 generic.go:356] "Generic (PLEG): container finished" podID="367da045-afa4-48c1-a117-25c808345068" containerID="d78e9c3360e40bb1035836cd09567203c9645881635a62cfa981fa80ed0cbdd0" exitCode=0 Jul 18 02:33:39.966235 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:39.965664 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7428-140098145593216" event={"ID":"367da045-afa4-48c1-a117-25c808345068","Type":"ContainerDied","Data":"d78e9c3360e40bb1035836cd09567203c9645881635a62cfa981fa80ed0cbdd0"} Jul 18 02:33:41.110987 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:41.110962 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145593216" Jul 18 02:33:41.130024 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:41.129996 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7428-140098145593216_367da045-afa4-48c1-a117-25c808345068/test-curl-7428-140098145593216/0.log" Jul 18 02:33:41.157484 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:41.157451 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hn7lf\" (UniqueName: \"kubernetes.io/projected/367da045-afa4-48c1-a117-25c808345068-kube-api-access-hn7lf\") pod \"367da045-afa4-48c1-a117-25c808345068\" (UID: \"367da045-afa4-48c1-a117-25c808345068\") " Jul 18 02:33:41.158055 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:41.158035 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7428-140098145593216"] Jul 18 02:33:41.159796 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:41.159768 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/367da045-afa4-48c1-a117-25c808345068-kube-api-access-hn7lf" (OuterVolumeSpecName: "kube-api-access-hn7lf") pod "367da045-afa4-48c1-a117-25c808345068" (UID: "367da045-afa4-48c1-a117-25c808345068"). InnerVolumeSpecName "kube-api-access-hn7lf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:33:41.161586 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:41.161561 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7428-140098145593216"] Jul 18 02:33:41.258697 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:41.258583 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn7lf\" (UniqueName: \"kubernetes.io/projected/367da045-afa4-48c1-a117-25c808345068-kube-api-access-hn7lf\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:33:41.978597 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:41.978565 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7428-140098145593216" Jul 18 02:33:41.978794 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:41.978612 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ab7591e1134e1b7ee8ad051ddc5d7dda8aefcc447b9ae40eda6db002203b8d9" Jul 18 02:33:41.992419 ip-10-0-132-151 kubenswrapper[2570]: E0718 02:33:41.992383 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7428-140098145593216\" is forbidden: User \"system:node:ip-10-0-132-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-151.ec2.internal' and this object" podUID="367da045-afa4-48c1-a117-25c808345068" pod="opendatahub/test-curl-7428-140098145593216" Jul 18 02:33:42.895093 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:33:42.895058 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="367da045-afa4-48c1-a117-25c808345068" path="/var/lib/kubelet/pods/367da045-afa4-48c1-a117-25c808345068/volumes" Jul 18 02:34:13.684922 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:13.684824 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl"] Jul 18 02:34:13.685603 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:13.685081 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" podUID="2555d1a2-0c25-4792-85f4-3b845f2aee5d" containerName="maas-api" containerID="cri-o://cd88dec291b1702abb8830109fb8b5e6103d27d568913632e8bfdd41813e4b7b" gracePeriod=30 Jul 18 02:34:14.147169 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:14.147138 2570 generic.go:356] "Generic (PLEG): container finished" podID="2555d1a2-0c25-4792-85f4-3b845f2aee5d" containerID="cd88dec291b1702abb8830109fb8b5e6103d27d568913632e8bfdd41813e4b7b" exitCode=0 Jul 18 02:34:14.147327 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:14.147209 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" event={"ID":"2555d1a2-0c25-4792-85f4-3b845f2aee5d","Type":"ContainerDied","Data":"cd88dec291b1702abb8830109fb8b5e6103d27d568913632e8bfdd41813e4b7b"} Jul 18 02:34:14.147327 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:14.147259 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" event={"ID":"2555d1a2-0c25-4792-85f4-3b845f2aee5d","Type":"ContainerDied","Data":"c4a5d2f64daa16ca4930477afe865bcadde146fa76e2f063a7f19d02958145fb"} Jul 18 02:34:14.147327 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:14.147283 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4a5d2f64daa16ca4930477afe865bcadde146fa76e2f063a7f19d02958145fb" Jul 18 02:34:14.148020 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:14.148003 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" Jul 18 02:34:14.150431 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:14.150414 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-484nb\" (UniqueName: \"kubernetes.io/projected/2555d1a2-0c25-4792-85f4-3b845f2aee5d-kube-api-access-484nb\") pod \"2555d1a2-0c25-4792-85f4-3b845f2aee5d\" (UID: \"2555d1a2-0c25-4792-85f4-3b845f2aee5d\") " Jul 18 02:34:14.150519 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:14.150507 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2555d1a2-0c25-4792-85f4-3b845f2aee5d-maas-api-tls\") pod \"2555d1a2-0c25-4792-85f4-3b845f2aee5d\" (UID: \"2555d1a2-0c25-4792-85f4-3b845f2aee5d\") " Jul 18 02:34:14.150557 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:14.150536 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2555d1a2-0c25-4792-85f4-3b845f2aee5d-openshift-service-ca\") pod \"2555d1a2-0c25-4792-85f4-3b845f2aee5d\" (UID: \"2555d1a2-0c25-4792-85f4-3b845f2aee5d\") " Jul 18 02:34:14.150948 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:14.150923 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2555d1a2-0c25-4792-85f4-3b845f2aee5d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2555d1a2-0c25-4792-85f4-3b845f2aee5d" (UID: "2555d1a2-0c25-4792-85f4-3b845f2aee5d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:34:14.152778 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:14.152747 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2555d1a2-0c25-4792-85f4-3b845f2aee5d-kube-api-access-484nb" (OuterVolumeSpecName: "kube-api-access-484nb") pod "2555d1a2-0c25-4792-85f4-3b845f2aee5d" (UID: "2555d1a2-0c25-4792-85f4-3b845f2aee5d"). InnerVolumeSpecName "kube-api-access-484nb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:34:14.153075 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:14.153054 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2555d1a2-0c25-4792-85f4-3b845f2aee5d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2555d1a2-0c25-4792-85f4-3b845f2aee5d" (UID: "2555d1a2-0c25-4792-85f4-3b845f2aee5d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:34:14.251950 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:14.251858 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2555d1a2-0c25-4792-85f4-3b845f2aee5d-maas-api-tls\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:34:14.251950 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:14.251890 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2555d1a2-0c25-4792-85f4-3b845f2aee5d-openshift-service-ca\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:34:14.251950 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:14.251900 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-484nb\" (UniqueName: \"kubernetes.io/projected/2555d1a2-0c25-4792-85f4-3b845f2aee5d-kube-api-access-484nb\") on node \"ip-10-0-132-151.ec2.internal\" DevicePath \"\"" Jul 18 02:34:15.152696 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:15.152587 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl" Jul 18 02:34:15.167810 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:15.167775 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl"] Jul 18 02:34:15.171471 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:15.171442 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-16c040-cdb5df84f-9jvwl"] Jul 18 02:34:16.894348 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:16.894315 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2555d1a2-0c25-4792-85f4-3b845f2aee5d" path="/var/lib/kubelet/pods/2555d1a2-0c25-4792-85f4-3b845f2aee5d/volumes" Jul 18 02:34:25.675798 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:25.675759 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-cb6d88699-8fjgj_bcc8d07c-e198-425c-9b69-266fdf3740b3/authorino/0.log" Jul 18 02:34:30.970460 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:30.970417 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gw8pr_51a99fed-12aa-4a7d-a4d0-238bbb5454b1/manager/0.log" Jul 18 02:34:31.094719 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:31.094686 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ffh2m_520b5ebf-6700-4b39-97c6-1c364f0b9987/manager/0.log" Jul 18 02:34:31.215373 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:31.215333 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74bf56b8d6-jk6xr_02f9098a-81c5-4b4b-8eaf-0fc55cbe7519/manager/0.log" Jul 18 02:34:31.329304 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:31.329271 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5w78s_4298078f-366e-4871-be68-fda580328746/server/0.log" Jul 18 02:34:31.454914 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:31.454883 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jzws6_e8f8faab-8b16-4476-a4e4-f9056bc8de4b/manager/0.log" Jul 18 02:34:31.839584 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:31.839553 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6667d74d96-gc6zv_a898a04a-8c9c-46f1-b225-f108ebd7fe1b/copy-manifests/0.log" Jul 18 02:34:31.847562 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:31.847536 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6667d74d96-gc6zv_a898a04a-8c9c-46f1-b225-f108ebd7fe1b/manager/0.log" Jul 18 02:34:32.180794 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:32.180710 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5fc78b4c74-sbwsh_807b01d3-a35e-40df-bbdc-7044cab2d0aa/maas-api/0.log" Jul 18 02:34:32.292976 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:32.292947 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29739000-hpbt4_a07bb45f-f1e2-4c7a-b338-dc74b3f6a56a/cleanup/0.log" Jul 18 02:34:33.873767 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:33.873731 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-cb6d88699-8fjgj_bcc8d07c-e198-425c-9b69-266fdf3740b3/authorino/0.log" Jul 18 02:34:33.990168 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:33.990132 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-kgpc4_49b16d2d-a19a-456c-9cdb-4bffe2097d5b/manager/0.log" Jul 18 02:34:34.099119 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:34.099087 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4nl9h_6b5b4a89-42d7-4f2d-b73a-decefd4973c2/manager/0.log" Jul 18 02:34:34.327820 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:34.327773 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-fgrxl_eb17e69e-d29d-4ec2-9443-2509b6b827e0/registry-server/0.log" Jul 18 02:34:34.454826 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:34.454774 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-dkwx8_5faad5dd-db71-43f3-8a67-2bc9b7555c67/manager/0.log" Jul 18 02:34:34.691260 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:34.691163 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vq7zv_56774121-f7ed-4af6-9179-e0203a081e7e/manager/0.log" Jul 18 02:34:35.022053 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:35.021962 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4xh8tj_53a4628d-9ba5-4d60-8a90-3046f2b24220/istio-proxy/0.log" Jul 18 02:34:35.352764 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:35.352736 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c44dbb944-trg6n_9d13fee3-dded-4137-b742-601a6ecb93ae/kube-auth-proxy/0.log" Jul 18 02:34:35.480520 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:35.480483 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gf7sm_be7b9a46-e566-4a97-a560-4d0e070934fe/istio-proxy/1.log" Jul 18 02:34:35.600741 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:35.600700 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xc626_5f603196-b92d-4b4f-adc1-34698e2cb4d0/payload-pre-processing/0.log" Jul 18 02:34:35.712504 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:35.712388 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-l2p85_d0fe1c0b-338a-437e-8f00-6d4c560a7067/payload-processing/0.log" Jul 18 02:34:36.266762 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:36.266687 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-c6nqg_0524fea7-2bce-4d9d-bea9-9b6a4cec78a4/main/0.log" Jul 18 02:34:36.489286 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:36.489256 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbzwphn_306e5cc7-cef8-49fc-a09a-f7a9010c2e9f/main/0.log" Jul 18 02:34:41.206738 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:41.206708 2570 scope.go:117] "RemoveContainer" containerID="cd88dec291b1702abb8830109fb8b5e6103d27d568913632e8bfdd41813e4b7b" Jul 18 02:34:44.968758 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:44.968727 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bdkrt_2693ad6a-f241-4165-8d2e-9aee83cc8ae8/global-pull-secret-syncer/0.log" Jul 18 02:34:45.133104 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:45.133072 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wj9lj_8af123e0-fca5-4d69-b7cd-dcfdc8e1750e/konnectivity-agent/0.log" Jul 18 02:34:45.153387 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:45.153352 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-151.ec2.internal_90ea7be93df65d66da652d9417af881e/haproxy/0.log" Jul 18 02:34:50.370885 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:50.370846 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-cb6d88699-8fjgj_bcc8d07c-e198-425c-9b69-266fdf3740b3/authorino/0.log" Jul 18 02:34:50.406417 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:50.406381 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-kgpc4_49b16d2d-a19a-456c-9cdb-4bffe2097d5b/manager/0.log" Jul 18 02:34:50.439088 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:50.439054 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4nl9h_6b5b4a89-42d7-4f2d-b73a-decefd4973c2/manager/0.log" Jul 18 02:34:50.536461 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:50.536418 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-fgrxl_eb17e69e-d29d-4ec2-9443-2509b6b827e0/registry-server/0.log" Jul 18 02:34:50.732226 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:50.732128 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-dkwx8_5faad5dd-db71-43f3-8a67-2bc9b7555c67/manager/0.log" Jul 18 02:34:50.866713 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:50.866678 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-vq7zv_56774121-f7ed-4af6-9179-e0203a081e7e/manager/0.log" Jul 18 02:34:53.128471 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:53.128431 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-rh4q8_e8e164cc-e6b4-4ebd-b122-e383719192d0/cluster-monitoring-operator/0.log" Jul 18 02:34:53.325012 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:53.324982 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-664cc87cf7-glvjn_971cbd12-3157-444a-b7a4-fbb12ff4db1c/metrics-server/0.log" Jul 18 02:34:53.370633 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:53.370606 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-bjmwc_3c0d1cfd-e2ad-4ba2-9bc1-e773278b161e/monitoring-plugin/0.log" Jul 18 02:34:53.418469 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:53.418388 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c68mx_31d64b4a-8386-4a23-be41-0df94f6cabf4/node-exporter/0.log" Jul 18 02:34:53.438733 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:53.438704 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c68mx_31d64b4a-8386-4a23-be41-0df94f6cabf4/kube-rbac-proxy/0.log" Jul 18 02:34:53.457594 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:53.457569 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-c68mx_31d64b4a-8386-4a23-be41-0df94f6cabf4/init-textfile/0.log" Jul 18 02:34:53.941500 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:53.941465 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-wjc9q_3150d481-c72f-4cca-b9ba-049e43ddd627/prometheus-operator-admission-webhook/0.log" Jul 18 02:34:55.347837 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.347793 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-b76q4_f2d2e933-102a-44e0-9813-093c6ff90178/networking-console-plugin/0.log" Jul 18 02:34:55.830715 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.830680 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww"] Jul 18 02:34:55.831127 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.831113 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="367da045-afa4-48c1-a117-25c808345068" containerName="test-curl-7428-140098145593216" Jul 18 02:34:55.831181 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.831129 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="367da045-afa4-48c1-a117-25c808345068" containerName="test-curl-7428-140098145593216" Jul 18 02:34:55.831181 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.831144 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2555d1a2-0c25-4792-85f4-3b845f2aee5d" containerName="maas-api" Jul 18 02:34:55.831181 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.831150 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2555d1a2-0c25-4792-85f4-3b845f2aee5d" containerName="maas-api" Jul 18 02:34:55.831275 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.831225 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2555d1a2-0c25-4792-85f4-3b845f2aee5d" containerName="maas-api" Jul 18 02:34:55.831275 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.831241 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="367da045-afa4-48c1-a117-25c808345068" containerName="test-curl-7428-140098145593216" Jul 18 02:34:55.834547 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.834522 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:55.836810 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.836788 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4d8sb\"/\"kube-root-ca.crt\"" Jul 18 02:34:55.837512 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.837491 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4d8sb\"/\"default-dockercfg-6ncht\"" Jul 18 02:34:55.837634 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.837531 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4d8sb\"/\"openshift-service-ca.crt\"" Jul 18 02:34:55.843146 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.843118 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww"] Jul 18 02:34:55.934972 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.934919 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1109714e-d7c8-4f26-bbd4-f3b4c410dd11-sys\") pod \"perf-node-gather-daemonset-r27ww\" (UID: \"1109714e-d7c8-4f26-bbd4-f3b4c410dd11\") " pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:55.934972 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.934987 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1109714e-d7c8-4f26-bbd4-f3b4c410dd11-podres\") pod \"perf-node-gather-daemonset-r27ww\" (UID: \"1109714e-d7c8-4f26-bbd4-f3b4c410dd11\") " pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:55.935238 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.935049 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1109714e-d7c8-4f26-bbd4-f3b4c410dd11-lib-modules\") pod \"perf-node-gather-daemonset-r27ww\" (UID: \"1109714e-d7c8-4f26-bbd4-f3b4c410dd11\") " pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:55.935238 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.935116 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtvwb\" (UniqueName: \"kubernetes.io/projected/1109714e-d7c8-4f26-bbd4-f3b4c410dd11-kube-api-access-vtvwb\") pod \"perf-node-gather-daemonset-r27ww\" (UID: \"1109714e-d7c8-4f26-bbd4-f3b4c410dd11\") " pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:55.935238 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.935142 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1109714e-d7c8-4f26-bbd4-f3b4c410dd11-proc\") pod \"perf-node-gather-daemonset-r27ww\" (UID: \"1109714e-d7c8-4f26-bbd4-f3b4c410dd11\") " pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:55.986257 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.986224 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/1.log" Jul 18 02:34:55.994665 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:55.994611 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-8m75k_d5439579-cb04-47d9-99a6-60c482c4afbd/console-operator/2.log" Jul 18 02:34:56.036333 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:56.036296 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1109714e-d7c8-4f26-bbd4-f3b4c410dd11-sys\") pod \"perf-node-gather-daemonset-r27ww\" (UID: \"1109714e-d7c8-4f26-bbd4-f3b4c410dd11\") " pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:56.036510 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:56.036343 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1109714e-d7c8-4f26-bbd4-f3b4c410dd11-podres\") pod \"perf-node-gather-daemonset-r27ww\" (UID: \"1109714e-d7c8-4f26-bbd4-f3b4c410dd11\") " pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:56.036510 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:56.036361 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1109714e-d7c8-4f26-bbd4-f3b4c410dd11-lib-modules\") pod \"perf-node-gather-daemonset-r27ww\" (UID: \"1109714e-d7c8-4f26-bbd4-f3b4c410dd11\") " pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:56.036510 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:56.036404 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtvwb\" (UniqueName: \"kubernetes.io/projected/1109714e-d7c8-4f26-bbd4-f3b4c410dd11-kube-api-access-vtvwb\") pod \"perf-node-gather-daemonset-r27ww\" (UID: \"1109714e-d7c8-4f26-bbd4-f3b4c410dd11\") " pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:56.036510 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:56.036427 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1109714e-d7c8-4f26-bbd4-f3b4c410dd11-proc\") pod \"perf-node-gather-daemonset-r27ww\" (UID: \"1109714e-d7c8-4f26-bbd4-f3b4c410dd11\") " pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:56.036510 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:56.036434 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1109714e-d7c8-4f26-bbd4-f3b4c410dd11-sys\") pod \"perf-node-gather-daemonset-r27ww\" (UID: \"1109714e-d7c8-4f26-bbd4-f3b4c410dd11\") " pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:56.036510 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:56.036472 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1109714e-d7c8-4f26-bbd4-f3b4c410dd11-podres\") pod \"perf-node-gather-daemonset-r27ww\" (UID: \"1109714e-d7c8-4f26-bbd4-f3b4c410dd11\") " pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:56.036510 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:56.036499 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1109714e-d7c8-4f26-bbd4-f3b4c410dd11-proc\") pod \"perf-node-gather-daemonset-r27ww\" (UID: \"1109714e-d7c8-4f26-bbd4-f3b4c410dd11\") " pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:56.036813 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:56.036555 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1109714e-d7c8-4f26-bbd4-f3b4c410dd11-lib-modules\") pod \"perf-node-gather-daemonset-r27ww\" (UID: \"1109714e-d7c8-4f26-bbd4-f3b4c410dd11\") " pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:56.043978 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:56.043944 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtvwb\" (UniqueName: \"kubernetes.io/projected/1109714e-d7c8-4f26-bbd4-f3b4c410dd11-kube-api-access-vtvwb\") pod \"perf-node-gather-daemonset-r27ww\" (UID: \"1109714e-d7c8-4f26-bbd4-f3b4c410dd11\") " pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:56.145243 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:56.145148 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:56.283943 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:56.283915 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww"] Jul 18 02:34:56.286013 ip-10-0-132-151 kubenswrapper[2570]: W0718 02:34:56.285986 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1109714e_d7c8_4f26_bbd4_f3b4c410dd11.slice/crio-04e5b39b31e60735a27cee4c669cbb165fce30d33c82dd2c861649af12a7a035 WatchSource:0}: Error finding container 04e5b39b31e60735a27cee4c669cbb165fce30d33c82dd2c861649af12a7a035: Status 404 returned error can't find the container with id 04e5b39b31e60735a27cee4c669cbb165fce30d33c82dd2c861649af12a7a035 Jul 18 02:34:56.371381 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:56.371345 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" event={"ID":"1109714e-d7c8-4f26-bbd4-f3b4c410dd11","Type":"ContainerStarted","Data":"04e5b39b31e60735a27cee4c669cbb165fce30d33c82dd2c861649af12a7a035"} Jul 18 02:34:56.540802 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:56.540769 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-kxctv_41d8c727-0f8f-4fdf-ae90-c10fd2caaaff/download-server/0.log" Jul 18 02:34:57.377177 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:57.377132 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" event={"ID":"1109714e-d7c8-4f26-bbd4-f3b4c410dd11","Type":"ContainerStarted","Data":"c42271e7bb9b3578fa12a9da7833d03a7ef603a4c488b42ae1cba3caa9c3121d"} Jul 18 02:34:57.377578 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:57.377253 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:34:57.391734 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:57.391629 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" podStartSLOduration=2.391612354 podStartE2EDuration="2.391612354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:34:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:34:57.390852722 +0000 UTC m=+2837.068276844" watchObservedRunningTime="2026-07-18 02:34:57.391612354 +0000 UTC m=+2837.069036462" Jul 18 02:34:58.109097 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:58.109068 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bzrjc_163a2e0f-86f1-4c36-ba43-6d097cbec349/dns/0.log" Jul 18 02:34:58.129051 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:58.129023 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bzrjc_163a2e0f-86f1-4c36-ba43-6d097cbec349/kube-rbac-proxy/0.log" Jul 18 02:34:58.246791 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:58.246759 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8xjzh_270e5f34-05a1-4e34-828a-2ade34235111/dns-node-resolver/0.log" Jul 18 02:34:58.839404 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:58.839373 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-566ms_abf94060-f8ec-42d6-8e79-2c6d3490ebf8/node-ca/0.log" Jul 18 02:34:59.817303 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:59.817273 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4xh8tj_53a4628d-9ba5-4d60-8a90-3046f2b24220/istio-proxy/0.log" Jul 18 02:34:59.979305 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:34:59.979273 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c44dbb944-trg6n_9d13fee3-dded-4137-b742-601a6ecb93ae/kube-auth-proxy/0.log" Jul 18 02:35:00.008818 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:00.008786 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gf7sm_be7b9a46-e566-4a97-a560-4d0e070934fe/istio-proxy/0.log" Jul 18 02:35:00.069305 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:00.069204 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gf7sm_be7b9a46-e566-4a97-a560-4d0e070934fe/istio-proxy/1.log" Jul 18 02:35:00.104069 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:00.104026 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xc626_5f603196-b92d-4b4f-adc1-34698e2cb4d0/payload-pre-processing/0.log" Jul 18 02:35:00.141036 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:00.140999 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-l2p85_d0fe1c0b-338a-437e-8f00-6d4c560a7067/payload-processing/0.log" Jul 18 02:35:00.844821 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:00.844791 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b78nn_f9231edc-64b9-43d0-b939-e9043f91e610/serve-healthcheck-canary/0.log" Jul 18 02:35:01.503623 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:01.503590 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kh7bp_9d0a5939-81a2-4ed5-a54a-84ed81158dd5/kube-rbac-proxy/0.log" Jul 18 02:35:01.524131 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:01.524103 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kh7bp_9d0a5939-81a2-4ed5-a54a-84ed81158dd5/exporter/0.log" Jul 18 02:35:01.545572 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:01.545538 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kh7bp_9d0a5939-81a2-4ed5-a54a-84ed81158dd5/extractor/0.log" Jul 18 02:35:03.392819 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:03.392790 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4d8sb/perf-node-gather-daemonset-r27ww" Jul 18 02:35:03.667409 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:03.667313 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gw8pr_51a99fed-12aa-4a7d-a4d0-238bbb5454b1/manager/0.log" Jul 18 02:35:03.991288 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:03.991186 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ffh2m_520b5ebf-6700-4b39-97c6-1c364f0b9987/manager/0.log" Jul 18 02:35:04.180203 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:04.180152 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74bf56b8d6-jk6xr_02f9098a-81c5-4b4b-8eaf-0fc55cbe7519/manager/0.log" Jul 18 02:35:04.200727 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:04.200699 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5w78s_4298078f-366e-4871-be68-fda580328746/server/0.log" Jul 18 02:35:04.307726 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:04.307691 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-jzws6_e8f8faab-8b16-4476-a4e4-f9056bc8de4b/manager/0.log" Jul 18 02:35:04.472550 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:04.472520 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6667d74d96-gc6zv_a898a04a-8c9c-46f1-b225-f108ebd7fe1b/manager/0.log" Jul 18 02:35:04.495057 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:04.495032 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6667d74d96-gc6zv_a898a04a-8c9c-46f1-b225-f108ebd7fe1b/copy-manifests/0.log" Jul 18 02:35:06.510339 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:06.510306 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4xh8tj_53a4628d-9ba5-4d60-8a90-3046f2b24220/istio-proxy/0.log" Jul 18 02:35:06.640552 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:06.640513 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c44dbb944-trg6n_9d13fee3-dded-4137-b742-601a6ecb93ae/kube-auth-proxy/0.log" Jul 18 02:35:06.660870 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:06.660826 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gf7sm_be7b9a46-e566-4a97-a560-4d0e070934fe/istio-proxy/0.log" Jul 18 02:35:06.721264 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:06.721233 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gf7sm_be7b9a46-e566-4a97-a560-4d0e070934fe/istio-proxy/1.log" Jul 18 02:35:06.749268 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:06.749236 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xc626_5f603196-b92d-4b4f-adc1-34698e2cb4d0/payload-pre-processing/0.log" Jul 18 02:35:06.785590 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:06.785489 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-l2p85_d0fe1c0b-338a-437e-8f00-6d4c560a7067/payload-processing/0.log" Jul 18 02:35:07.972659 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:07.972599 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4xh8tj_53a4628d-9ba5-4d60-8a90-3046f2b24220/istio-proxy/0.log" Jul 18 02:35:08.122526 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:08.122496 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c44dbb944-trg6n_9d13fee3-dded-4137-b742-601a6ecb93ae/kube-auth-proxy/0.log" Jul 18 02:35:08.149722 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:08.149684 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gf7sm_be7b9a46-e566-4a97-a560-4d0e070934fe/istio-proxy/0.log" Jul 18 02:35:08.210077 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:08.210036 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-gf7sm_be7b9a46-e566-4a97-a560-4d0e070934fe/istio-proxy/1.log" Jul 18 02:35:08.242363 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:08.242266 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xc626_5f603196-b92d-4b4f-adc1-34698e2cb4d0/payload-pre-processing/0.log" Jul 18 02:35:08.274828 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:08.274801 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-l2p85_d0fe1c0b-338a-437e-8f00-6d4c560a7067/payload-processing/0.log" Jul 18 02:35:12.403118 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:12.403087 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5lxc9_d53a7743-60f0-4308-8165-6b27513efeb4/kube-multus-additional-cni-plugins/0.log" Jul 18 02:35:12.422069 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:12.422034 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5lxc9_d53a7743-60f0-4308-8165-6b27513efeb4/egress-router-binary-copy/0.log" Jul 18 02:35:12.440669 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:12.440630 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5lxc9_d53a7743-60f0-4308-8165-6b27513efeb4/cni-plugins/0.log" Jul 18 02:35:12.458693 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:12.458663 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5lxc9_d53a7743-60f0-4308-8165-6b27513efeb4/bond-cni-plugin/0.log" Jul 18 02:35:12.476611 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:12.476586 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5lxc9_d53a7743-60f0-4308-8165-6b27513efeb4/routeoverride-cni/0.log" Jul 18 02:35:12.494481 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:12.494440 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5lxc9_d53a7743-60f0-4308-8165-6b27513efeb4/whereabouts-cni-bincopy/0.log" Jul 18 02:35:12.513513 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:12.513481 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5lxc9_d53a7743-60f0-4308-8165-6b27513efeb4/whereabouts-cni/0.log" Jul 18 02:35:12.749382 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:12.749300 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-xbkpg_8baf11f1-2c27-4a7c-97c1-73f26856fe39/kube-multus/0.log" Jul 18 02:35:12.804922 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:12.804889 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5nqdq_bf7391f7-3b4a-4eb5-ab3a-b540543d4907/network-metrics-daemon/0.log" Jul 18 02:35:12.825097 ip-10-0-132-151 kubenswrapper[2570]: I0718 02:35:12.825066 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5nqdq_bf7391f7-3b4a-4eb5-ab3a-b540543d4907/kube-rbac-proxy/0.log"