Jul 18 01:48:09.667386 ip-10-0-137-108 systemd[1]: Starting Kubernetes Kubelet... Jul 18 01:48:09.964268 ip-10-0-137-108 kubenswrapper[2548]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 01:48:09.964268 ip-10-0-137-108 kubenswrapper[2548]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 18 01:48:09.964268 ip-10-0-137-108 kubenswrapper[2548]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 01:48:09.964268 ip-10-0-137-108 kubenswrapper[2548]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 18 01:48:09.964268 ip-10-0-137-108 kubenswrapper[2548]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 18 01:48:09.964956 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.964880 2548 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 18 01:48:09.969153 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969138 2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:48:09.969153 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969153 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969157 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969161 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969164 2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969167 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969170 2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969173 2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969175 2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969178 2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969181 2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969183 2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969186 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969189 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969191 2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969194 2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969197 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969199 2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969203 2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969205 2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969208 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:48:09.969224 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969211 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969213 2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969216 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969218 2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969221 2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969223 2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969226 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969229 2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969231 2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969234 2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969237 2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969240 2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969243 2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969246 2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969248 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969251 2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969254 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969256 2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969259 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969263 2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:48:09.969702 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969267 2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969270 2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969273 2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969276 2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969279 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969282 2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969285 2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969288 2548 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969290 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969299 2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969303 2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969306 2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969308 2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969311 2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969314 2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969317 2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969319 2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969322 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969325 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969328 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:48:09.970171 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969330 2548 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969332 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969335 2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969338 2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969340 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969343 2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969346 2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969348 2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969351 2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969354 2548 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969357 2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969361 2548 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969364 2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969368 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969370 2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969373 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969376 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969378 2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969381 2548 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:48:09.970679 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969383 2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969386 2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969389 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969392 2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969395 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969397 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969400 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969415 2548 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969418 2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969421 2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969425 2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969429 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969432 2548 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969435 2548 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969438 2548 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969441 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969443 2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969446 2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.969449 2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970122 2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:48:09.971145 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970129 2548 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970132 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970134 2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970137 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970140 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970142 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970145 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970148 2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970151 2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970154 2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970156 2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970159 2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970161 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970164 2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970167 2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970170 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970173 2548 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970176 2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970179 2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970183 2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:48:09.971672 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970186 2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970189 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970192 2548 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970194 2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970197 2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970200 2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970202 2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970205 2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970207 2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970210 2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970213 2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970215 2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970218 2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970222 2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970226 2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970229 2548 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970232 2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970234 2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970237 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:48:09.972154 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970240 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970243 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970246 2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970248 2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970251 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970253 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970255 2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970258 2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970261 2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970264 2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970268 2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970271 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970273 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970276 2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970279 2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970281 2548 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970284 2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970287 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970289 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970292 2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:48:09.972639 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970294 2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970297 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970299 2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970302 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970305 2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970307 2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970310 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970312 2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970315 2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970317 2548 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970320 2548 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970322 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970325 2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970327 2548 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970330 2548 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970333 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970336 2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970339 2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970342 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970345 2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:48:09.973120 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970347 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970350 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970353 2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970355 2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970361 2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970363 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970366 2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970369 2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970371 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970375 2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970377 2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970380 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970383 2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970385 2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970387 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970392 2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970395 2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970398 2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970401 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:48:09.973602 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970760 2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970765 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970768 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970771 2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970774 2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970777 2548 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970779 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970782 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970785 2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970787 2548 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970790 2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970792 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970795 2548 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970798 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970800 2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970803 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970806 2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970810 2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970812 2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970815 2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:48:09.974049 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970818 2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970821 2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970823 2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970826 2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970828 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970833 2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970836 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970839 2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970842 2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970845 2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970848 2548 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970851 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970854 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970856 2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970859 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970862 2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970864 2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970867 2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970869 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:48:09.974608 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970872 2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970874 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970877 2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970879 2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970882 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970884 2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970887 2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970890 2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970892 2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970897 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970900 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970903 2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970905 2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970908 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970910 2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970912 2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970915 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970918 2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970920 2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:48:09.975313 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970923 2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970926 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970928 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970931 2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970934 2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970936 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970938 2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970941 2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970944 2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970947 2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970949 2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970953 2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970957 2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970960 2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970963 2548 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970965 2548 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970969 2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970972 2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970976 2548 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970981 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:48:09.975823 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970984 2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970987 2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970990 2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970994 2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.970996 2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971000 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971002 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971005 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971007 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971010 2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971013 2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971015 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971018 2548 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971021 2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971024 2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971027 2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971029 2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971032 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971034 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971037 2548 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:48:09.976311 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971040 2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971109 2548 flags.go:64] FLAG: --address="0.0.0.0" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971115 2548 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971121 2548 flags.go:64] FLAG: --anonymous-auth="true" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971125 2548 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971129 2548 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971134 2548 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971140 2548 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971147 2548 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971152 2548 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971157 2548 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971163 2548 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971167 2548 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971170 2548 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971173 2548 flags.go:64] FLAG: --cgroup-root="" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971177 2548 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971180 2548 flags.go:64] FLAG: --client-ca-file="" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971183 2548 flags.go:64] FLAG: --cloud-provider="external" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971186 2548 flags.go:64] FLAG: --cluster-dns="[]" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971191 2548 flags.go:64] FLAG: --cluster-domain="" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971194 2548 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971197 2548 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971200 2548 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971203 2548 flags.go:64] FLAG: --container-log-max-files="5" Jul 18 01:48:09.976854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971207 2548 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971210 2548 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971213 2548 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971217 2548 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971220 2548 flags.go:64] FLAG: --contention-profiling="false" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971223 2548 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971225 2548 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971229 2548 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971232 2548 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971235 2548 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971238 2548 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971241 2548 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971244 2548 flags.go:64] FLAG: --enable-load-reader="false" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971247 2548 flags.go:64] FLAG: --enable-server="true" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971250 2548 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971254 2548 flags.go:64] FLAG: --event-burst="100" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971257 2548 flags.go:64] FLAG: --event-qps="50" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971260 2548 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971263 2548 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971267 2548 flags.go:64] FLAG: --eviction-hard="" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971270 2548 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971273 2548 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971276 2548 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971279 2548 flags.go:64] FLAG: --eviction-soft="" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971283 2548 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 18 01:48:09.977481 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971286 2548 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971289 2548 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971292 2548 flags.go:64] FLAG: --experimental-mounter-path="" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971295 2548 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971298 2548 flags.go:64] FLAG: --fail-swap-on="true" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971301 2548 flags.go:64] FLAG: --feature-gates="" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971304 2548 flags.go:64] FLAG: --file-check-frequency="20s" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971307 2548 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971311 2548 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971313 2548 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971316 2548 flags.go:64] FLAG: --healthz-port="10248" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971319 2548 flags.go:64] FLAG: --help="false" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971322 2548 flags.go:64] FLAG: --hostname-override="ip-10-0-137-108.ec2.internal" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971325 2548 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971328 2548 flags.go:64] FLAG: --http-check-frequency="20s" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971331 2548 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971334 2548 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971337 2548 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971340 2548 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971343 2548 flags.go:64] FLAG: --image-service-endpoint="" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971346 2548 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971349 2548 flags.go:64] FLAG: --kube-api-burst="100" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971351 2548 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971355 2548 flags.go:64] FLAG: --kube-api-qps="50" Jul 18 01:48:09.978100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971358 2548 flags.go:64] FLAG: --kube-reserved="" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971361 2548 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971365 2548 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971368 2548 flags.go:64] FLAG: --kubelet-cgroups="" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971371 2548 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971374 2548 flags.go:64] FLAG: --lock-file="" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971377 2548 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971380 2548 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971383 2548 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971388 2548 flags.go:64] FLAG: --log-json-split-stream="false" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971391 2548 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971394 2548 flags.go:64] FLAG: --log-text-split-stream="false" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971397 2548 flags.go:64] FLAG: --logging-format="text" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971400 2548 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971415 2548 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971419 2548 flags.go:64] FLAG: --manifest-url="" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971422 2548 flags.go:64] FLAG: --manifest-url-header="" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971429 2548 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971431 2548 flags.go:64] FLAG: --max-open-files="1000000" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971436 2548 flags.go:64] FLAG: --max-pods="110" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971438 2548 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971442 2548 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971444 2548 flags.go:64] FLAG: --memory-manager-policy="None" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971447 2548 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971450 2548 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 18 01:48:09.978756 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971453 2548 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971456 2548 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971463 2548 flags.go:64] FLAG: --node-status-max-images="50" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971465 2548 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971468 2548 flags.go:64] FLAG: --oom-score-adj="-999" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971471 2548 flags.go:64] FLAG: --pod-cidr="" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971474 2548 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971480 2548 flags.go:64] FLAG: --pod-manifest-path="" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971483 2548 flags.go:64] FLAG: --pod-max-pids="-1" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971486 2548 flags.go:64] FLAG: --pods-per-core="0" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971490 2548 flags.go:64] FLAG: --port="10250" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971493 2548 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971496 2548 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c6bde19abd503acb" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971499 2548 flags.go:64] FLAG: --qos-reserved="" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971502 2548 flags.go:64] FLAG: --read-only-port="10255" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971505 2548 flags.go:64] FLAG: --register-node="true" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971508 2548 flags.go:64] FLAG: --register-with-taints="" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971512 2548 flags.go:64] FLAG: --registry-burst="10" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971515 2548 flags.go:64] FLAG: --registry-qps="5" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971518 2548 flags.go:64] FLAG: --reserved-cpus="" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971521 2548 flags.go:64] FLAG: --reserved-memory="" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971524 2548 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971527 2548 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971531 2548 flags.go:64] FLAG: --rotate-certificates="false" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971533 2548 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 18 01:48:09.979343 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971536 2548 flags.go:64] FLAG: --runonce="false" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971539 2548 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971542 2548 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971545 2548 flags.go:64] FLAG: --seccomp-default="false" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971548 2548 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971551 2548 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971554 2548 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971557 2548 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971559 2548 flags.go:64] FLAG: --storage-driver-password="root" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971562 2548 flags.go:64] FLAG: --storage-driver-secure="false" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971565 2548 flags.go:64] FLAG: --storage-driver-table="stats" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971568 2548 flags.go:64] FLAG: --storage-driver-user="root" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971571 2548 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971574 2548 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971577 2548 flags.go:64] FLAG: --system-cgroups="" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971579 2548 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971584 2548 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971589 2548 flags.go:64] FLAG: --tls-cert-file="" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971592 2548 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971598 2548 flags.go:64] FLAG: --tls-min-version="" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971601 2548 flags.go:64] FLAG: --tls-private-key-file="" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971604 2548 flags.go:64] FLAG: --topology-manager-policy="none" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971607 2548 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971610 2548 flags.go:64] FLAG: --topology-manager-scope="container" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971613 2548 flags.go:64] FLAG: --v="2" Jul 18 01:48:09.979950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971617 2548 flags.go:64] FLAG: --version="false" Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971620 2548 flags.go:64] FLAG: --vmodule="" Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971624 2548 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.971628 2548 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971742 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971746 2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971749 2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971753 2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971755 2548 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971758 2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971761 2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971764 2548 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971767 2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971770 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971772 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971775 2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971778 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971780 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971783 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971786 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:48:09.980579 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971788 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971791 2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971794 2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971796 2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971800 2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971802 2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971806 2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971809 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971812 2548 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971814 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971817 2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971819 2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971822 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971825 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971827 2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971831 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971834 2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971836 2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971839 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971842 2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:48:09.981054 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971844 2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971847 2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971849 2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971852 2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971854 2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971857 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971859 2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971862 2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971864 2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971867 2548 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971869 2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971872 2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971875 2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971877 2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971880 2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971883 2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971887 2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971891 2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971896 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:48:09.981546 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971899 2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971902 2548 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971905 2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971908 2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971911 2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971914 2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971917 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971919 2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971922 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971926 2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971929 2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971932 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971935 2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971938 2548 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971941 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971943 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971946 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971949 2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971952 2548 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971955 2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:48:09.982010 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971957 2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971961 2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971964 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971967 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971970 2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971974 2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971976 2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971979 2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971982 2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971986 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971989 2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971993 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971995 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.971998 2548 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.972000 2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.972003 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.972006 2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.972009 2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.972011 2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:48:09.982496 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.972014 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:48:09.982945 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.972016 2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:48:09.982945 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.972019 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:48:09.982945 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.972023 2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:48:09.982945 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.972027 2548 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:48:09.982945 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.972033 2548 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 01:48:09.982945 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.978530 2548 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 18 01:48:09.982945 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.978544 2548 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 18 01:48:09.982945 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.978564 2548 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 18 01:48:09.982945 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.978569 2548 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 18 01:48:09.982945 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978623 2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:48:09.982945 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978628 2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:48:09.982945 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978631 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:48:09.982945 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978634 2548 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:48:09.982945 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978637 2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978640 2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978643 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978646 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978649 2548 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978652 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978655 2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978657 2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978661 2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978665 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978669 2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978672 2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978675 2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978677 2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978680 2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978683 2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978686 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978689 2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978691 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:48:09.983288 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978694 2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978696 2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978699 2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978702 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978704 2548 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978707 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978711 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978714 2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978716 2548 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978719 2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978721 2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978724 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978726 2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978729 2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978732 2548 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978734 2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978737 2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978739 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978742 2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978744 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:48:09.983811 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978747 2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978749 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978752 2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978755 2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978757 2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978760 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978763 2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978766 2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978768 2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978772 2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978776 2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978779 2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978781 2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978784 2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978786 2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978789 2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978792 2548 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978794 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978797 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978800 2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:48:09.984286 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978803 2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978805 2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978808 2548 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978810 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978813 2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978815 2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978818 2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978820 2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978823 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978825 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978828 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978831 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978833 2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978836 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978838 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978843 2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978847 2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978850 2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978853 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:48:09.984796 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978855 2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:48:09.985243 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978858 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:48:09.985243 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978861 2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:48:09.985243 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978864 2548 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:48:09.985243 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978867 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:48:09.985243 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978870 2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:48:09.985243 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978873 2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:48:09.985243 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978876 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:48:09.985243 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978879 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:48:09.985243 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978881 2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:48:09.985243 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978884 2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:48:09.985243 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978887 2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:48:09.985243 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978890 2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:48:09.985243 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978892 2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:48:09.985243 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978895 2548 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:48:09.985243 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978897 2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:48:09.985243 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.978900 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:48:09.985754 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.978905 2548 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 01:48:09.985754 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979013 2548 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 18 01:48:09.985754 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979018 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 18 01:48:09.985754 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979021 2548 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 18 01:48:09.985754 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979024 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 18 01:48:09.985754 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979026 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 18 01:48:09.985754 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979029 2548 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 18 01:48:09.985754 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979032 2548 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 18 01:48:09.985754 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979034 2548 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 18 01:48:09.985754 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979037 2548 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 18 01:48:09.985754 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979039 2548 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 18 01:48:09.985754 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979042 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 18 01:48:09.985754 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979044 2548 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 18 01:48:09.985754 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979048 2548 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 18 01:48:09.985754 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979051 2548 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979053 2548 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979056 2548 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979058 2548 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979061 2548 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979063 2548 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979066 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979068 2548 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979071 2548 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979074 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979077 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979079 2548 feature_gate.go:328] unrecognized feature gate: Example Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979082 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979085 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979088 2548 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979090 2548 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979093 2548 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979097 2548 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979101 2548 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 18 01:48:09.986111 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979104 2548 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979107 2548 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979109 2548 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979112 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979115 2548 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979118 2548 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979120 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979123 2548 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979127 2548 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979130 2548 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979133 2548 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979136 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979139 2548 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979143 2548 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979145 2548 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979148 2548 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979151 2548 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979153 2548 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979156 2548 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 18 01:48:09.986598 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979158 2548 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979162 2548 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979166 2548 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979169 2548 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979171 2548 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979174 2548 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979177 2548 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979180 2548 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979182 2548 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979185 2548 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979187 2548 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979190 2548 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979192 2548 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979195 2548 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979198 2548 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979200 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979202 2548 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979205 2548 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979208 2548 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 18 01:48:09.987039 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979210 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979212 2548 feature_gate.go:328] unrecognized feature gate: Example2 Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979215 2548 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979217 2548 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979220 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979223 2548 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979225 2548 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979227 2548 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979231 2548 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979233 2548 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979236 2548 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979238 2548 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979241 2548 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979243 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979246 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979248 2548 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979255 2548 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979258 2548 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979261 2548 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979264 2548 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 18 01:48:09.987504 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979266 2548 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 18 01:48:09.987970 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979269 2548 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 18 01:48:09.987970 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979272 2548 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 18 01:48:09.987970 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979274 2548 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 18 01:48:09.987970 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979277 2548 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 18 01:48:09.987970 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979279 2548 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 18 01:48:09.987970 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979282 2548 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 18 01:48:09.987970 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979285 2548 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 18 01:48:09.987970 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:09.979287 2548 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 18 01:48:09.987970 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.979292 2548 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 18 01:48:09.987970 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.979380 2548 server.go:962] "Client rotation is on, will bootstrap in background" Jul 18 01:48:09.987970 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.981302 2548 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 18 01:48:09.987970 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.982418 2548 server.go:1019] "Starting client certificate rotation" Jul 18 01:48:09.987970 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.982514 2548 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 01:48:09.987970 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.982547 2548 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 18 01:48:09.988309 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.984422 2548 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 01:48:09.988431 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.988414 2548 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 18 01:48:09.999538 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:09.999520 2548 log.go:25] "Validated CRI v1 runtime API" Jul 18 01:48:10.001626 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.001608 2548 log.go:25] "Validated CRI v1 image API" Jul 18 01:48:10.002985 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.002963 2548 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 18 01:48:10.006534 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.006513 2548 fs.go:135] Filesystem UUIDs: map[7808142e-b819-49a3-9402-e4b339f36d18:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c6212f24-0955-4b22-b18b-a873185d8aa4:/dev/nvme0n1p4] Jul 18 01:48:10.006637 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.006534 2548 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 18 01:48:10.010456 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.010434 2548 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 01:48:10.012155 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.012049 2548 manager.go:216] Machine: {Timestamp:2026-07-18 01:48:10.010987937 +0000 UTC m=+0.253102612 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099796 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25ad3d9e1f3f76e8d10e6f7b6826a2 SystemUUID:ec25ad3d-9e1f-3f76-e8d1-0e6f7b6826a2 BootID:4032eff5-724d-4163-ae10-db37ad2d9bba Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:56:56:64:10:b3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:56:56:64:10:b3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:99:57:76:57:b7 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 18 01:48:10.012155 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.012149 2548 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 18 01:48:10.012308 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.012241 2548 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 18 01:48:10.014792 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.014769 2548 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 18 01:48:10.014915 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.014793 2548 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-108.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 18 01:48:10.015866 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.015856 2548 topology_manager.go:138] "Creating topology manager with none policy" Jul 18 01:48:10.015903 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.015868 2548 container_manager_linux.go:309] "Creating device plugin manager" Jul 18 01:48:10.015903 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.015875 2548 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 01:48:10.016542 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.016532 2548 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 18 01:48:10.017851 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.017841 2548 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 18 01:48:10.021490 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.021479 2548 state_mem.go:36] "Initialized new in-memory state store" Jul 18 01:48:10.021603 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.021593 2548 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 18 01:48:10.021634 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.021624 2548 kubelet.go:486] "Attempting to sync node with API server" Jul 18 01:48:10.021661 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.021635 2548 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 18 01:48:10.021661 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.021649 2548 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 18 01:48:10.021661 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.021657 2548 kubelet.go:398] "Adding apiserver pod source" Jul 18 01:48:10.021777 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.021665 2548 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 18 01:48:10.021978 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.021966 2548 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 01:48:10.022039 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.021984 2548 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 18 01:48:10.024226 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.024212 2548 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 18 01:48:10.025781 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.025768 2548 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 18 01:48:10.025834 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.025798 2548 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 18 01:48:10.026818 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.026807 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 18 01:48:10.026859 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.026828 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 18 01:48:10.026859 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.026838 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 18 01:48:10.026859 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.026845 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 18 01:48:10.026859 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.026853 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 18 01:48:10.026859 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.026859 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 18 01:48:10.026984 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.026867 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 18 01:48:10.026984 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.026872 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 18 01:48:10.026984 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.026880 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 18 01:48:10.026984 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.026890 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 18 01:48:10.026984 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.026899 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 18 01:48:10.026984 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.026905 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 18 01:48:10.027977 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.027968 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 18 01:48:10.028010 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.027978 2548 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 18 01:48:10.030925 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.030914 2548 server.go:1268] "Started kubelet" Jul 18 01:48:10.031098 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.030999 2548 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 18 01:48:10.031161 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.031116 2548 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 18 01:48:10.031523 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.031018 2548 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 18 01:48:10.031853 ip-10-0-137-108 systemd[1]: Started Kubernetes Kubelet. Jul 18 01:48:10.032009 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:10.031915 2548 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-108.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 18 01:48:10.032072 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.032049 2548 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-108.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 18 01:48:10.032151 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:10.032132 2548 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 18 01:48:10.032806 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.032763 2548 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 18 01:48:10.036725 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.036705 2548 server.go:310] "Adding debug handlers to kubelet server" Jul 18 01:48:10.038364 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:10.038335 2548 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 18 01:48:10.039689 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:10.037782 2548 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-108.ec2.internal.18c33f21651055fb default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-108.ec2.internal,UID:ip-10-0-137-108.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-108.ec2.internal,},FirstTimestamp:2026-07-18 01:48:10.030888443 +0000 UTC m=+0.273003117,LastTimestamp:2026-07-18 01:48:10.030888443 +0000 UTC m=+0.273003117,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-108.ec2.internal,}" Jul 18 01:48:10.041487 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.041462 2548 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 18 01:48:10.041565 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.041462 2548 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 18 01:48:10.041629 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:10.041569 2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-108.ec2.internal\" not found" Jul 18 01:48:10.042035 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.042007 2548 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 18 01:48:10.042155 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.042144 2548 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 18 01:48:10.042286 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.042043 2548 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 18 01:48:10.042384 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.042321 2548 factory.go:55] Registering systemd factory Jul 18 01:48:10.042384 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.042336 2548 factory.go:223] Registration of the systemd container factory successfully Jul 18 01:48:10.042384 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.042367 2548 reconstruct.go:97] "Volume reconstruction finished" Jul 18 01:48:10.042384 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.042375 2548 reconciler.go:29] "Reconciler: start to sync state" Jul 18 01:48:10.042569 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.042552 2548 factory.go:153] Registering CRI-O factory Jul 18 01:48:10.042569 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.042571 2548 factory.go:223] Registration of the crio container factory successfully Jul 18 01:48:10.042765 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.042616 2548 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 18 01:48:10.042765 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.042634 2548 factory.go:103] Registering Raw factory Jul 18 01:48:10.042765 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.042644 2548 manager.go:1196] Started watching for new ooms in manager Jul 18 01:48:10.043175 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.043159 2548 manager.go:349] Starting recovery of all containers Jul 18 01:48:10.051728 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:10.051594 2548 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 18 01:48:10.051728 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:10.051588 2548 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-108.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 18 01:48:10.052702 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:10.052686 2548 manager.go:354] Recovery completed Jul 18 01:48:11.443758 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:11.443734 2548 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": readdirent /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 18 01:48:11.446772 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.446760 2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:48:11.448533 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.448504 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:48:11.448597 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.448547 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:48:11.448597 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.448562 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:48:11.449029 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.449013 2548 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7gpkd" Jul 18 01:48:11.449139 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.449120 2548 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 18 01:48:11.449139 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.449137 2548 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 18 01:48:11.449267 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.449140 2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:48:11.449267 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.449155 2548 state_mem.go:36] "Initialized new in-memory state store" Jul 18 01:48:11.449921 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.449904 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:48:11.449982 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.449935 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:48:11.449982 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.449949 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:48:11.451424 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.451412 2548 policy_none.go:49] "None policy: Start" Jul 18 01:48:11.451471 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.451428 2548 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 18 01:48:11.451471 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.451438 2548 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 18 01:48:11.453361 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.453350 2548 policy_none.go:47] "Start" Jul 18 01:48:11.458907 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.458891 2548 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7gpkd" Jul 18 01:48:11.459433 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:11.459376 2548 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-108.ec2.internal.18c33f21b98fe252 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-108.ec2.internal,UID:ip-10-0-137-108.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-108.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-108.ec2.internal,},FirstTimestamp:2026-07-18 01:48:11.448533586 +0000 UTC m=+0.301276925,LastTimestamp:2026-07-18 01:48:11.448533586 +0000 UTC m=+0.301276925,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-108.ec2.internal,}" Jul 18 01:48:11.502680 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.502663 2548 manager.go:337] "Starting Device Plugin manager" Jul 18 01:48:11.502769 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:11.502692 2548 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 18 01:48:11.502769 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.502702 2548 server.go:87] "Starting device plugin registration server" Jul 18 01:48:11.502999 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.502929 2548 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 18 01:48:11.502999 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.502939 2548 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 18 01:48:11.526603 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.503150 2548 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 18 01:48:11.526603 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.503232 2548 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 18 01:48:11.526603 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.503240 2548 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 18 01:48:11.526603 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:11.503674 2548 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 18 01:48:11.526603 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:11.503718 2548 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-108.ec2.internal\" not found" Jul 18 01:48:11.541124 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.541097 2548 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 18 01:48:11.542192 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.542166 2548 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 18 01:48:11.542260 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.542219 2548 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 18 01:48:11.542260 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.542244 2548 kubelet.go:2448] "Starting kubelet main sync loop" Jul 18 01:48:11.542339 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:11.542283 2548 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 18 01:48:11.544732 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.544716 2548 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:48:11.603855 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.603806 2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:48:11.604814 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.604800 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:48:11.604896 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.604834 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:48:11.604896 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.604849 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:48:11.604896 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.604877 2548 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-108.ec2.internal" Jul 18 01:48:11.614183 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.614166 2548 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-108.ec2.internal" Jul 18 01:48:11.614251 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:11.614185 2548 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-108.ec2.internal\": node \"ip-10-0-137-108.ec2.internal\" not found" Jul 18 01:48:11.637550 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:11.637527 2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-108.ec2.internal\" not found" Jul 18 01:48:11.642689 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.642664 2548 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal"] Jul 18 01:48:11.642768 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.642738 2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:48:11.644004 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.643990 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:48:11.644091 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.644020 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:48:11.644091 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.644034 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:48:11.646374 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.646360 2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:48:11.646484 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.646469 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 18 01:48:11.646533 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.646499 2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:48:11.647105 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.647089 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:48:11.647186 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.647096 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:48:11.647186 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.647150 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:48:11.647186 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.647162 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:48:11.647186 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.647120 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:48:11.647186 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.647187 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:48:11.649480 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.649465 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" Jul 18 01:48:11.649543 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.649490 2548 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 18 01:48:11.650239 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.650223 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientMemory" Jul 18 01:48:11.650328 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.650249 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasNoDiskPressure" Jul 18 01:48:11.650328 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.650265 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeHasSufficientPID" Jul 18 01:48:11.673394 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:11.673372 2548 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-108.ec2.internal\" not found" node="ip-10-0-137-108.ec2.internal" Jul 18 01:48:11.677637 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:11.677621 2548 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-108.ec2.internal\" not found" node="ip-10-0-137-108.ec2.internal" Jul 18 01:48:11.733278 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.733254 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ebbea173dadad9136c4ee6a95a9762e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal\" (UID: \"1ebbea173dadad9136c4ee6a95a9762e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 18 01:48:11.733278 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.733277 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d448a47aa0f5878864b37d16dc8c112-config\") pod \"kube-apiserver-proxy-ip-10-0-137-108.ec2.internal\" (UID: \"3d448a47aa0f5878864b37d16dc8c112\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" Jul 18 01:48:11.733388 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.733294 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ebbea173dadad9136c4ee6a95a9762e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal\" (UID: \"1ebbea173dadad9136c4ee6a95a9762e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 18 01:48:11.738365 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:11.738349 2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-108.ec2.internal\" not found" Jul 18 01:48:11.833583 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.833560 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ebbea173dadad9136c4ee6a95a9762e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal\" (UID: \"1ebbea173dadad9136c4ee6a95a9762e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 18 01:48:11.833667 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.833585 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d448a47aa0f5878864b37d16dc8c112-config\") pod \"kube-apiserver-proxy-ip-10-0-137-108.ec2.internal\" (UID: \"3d448a47aa0f5878864b37d16dc8c112\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" Jul 18 01:48:11.833667 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.833601 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ebbea173dadad9136c4ee6a95a9762e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal\" (UID: \"1ebbea173dadad9136c4ee6a95a9762e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 18 01:48:11.833667 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.833635 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1ebbea173dadad9136c4ee6a95a9762e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal\" (UID: \"1ebbea173dadad9136c4ee6a95a9762e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 18 01:48:11.833667 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.833646 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ebbea173dadad9136c4ee6a95a9762e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal\" (UID: \"1ebbea173dadad9136c4ee6a95a9762e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 18 01:48:11.833667 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.833659 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3d448a47aa0f5878864b37d16dc8c112-config\") pod \"kube-apiserver-proxy-ip-10-0-137-108.ec2.internal\" (UID: \"3d448a47aa0f5878864b37d16dc8c112\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" Jul 18 01:48:11.838660 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:11.838645 2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-108.ec2.internal\" not found" Jul 18 01:48:11.939453 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:11.939402 2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-108.ec2.internal\" not found" Jul 18 01:48:11.976565 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.976546 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 18 01:48:11.980107 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:11.980092 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" Jul 18 01:48:12.039764 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:12.039741 2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-108.ec2.internal\" not found" Jul 18 01:48:12.140203 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:12.140185 2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-108.ec2.internal\" not found" Jul 18 01:48:12.240770 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:12.240720 2548 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-108.ec2.internal\" not found" Jul 18 01:48:12.251414 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.251400 2548 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:48:12.331318 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.331297 2548 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" Jul 18 01:48:12.349136 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.349120 2548 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 01:48:12.350514 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.350502 2548 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" Jul 18 01:48:12.360105 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.360088 2548 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 18 01:48:12.371203 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.371186 2548 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 18 01:48:12.371287 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.371272 2548 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 18 01:48:12.411897 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.411875 2548 apiserver.go:52] "Watching apiserver" Jul 18 01:48:12.422656 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.422636 2548 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 18 01:48:12.423694 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.423673 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9","openshift-dns/node-resolver-bwx4x","openshift-multus/multus-additional-cni-plugins-hvmsl","openshift-multus/network-metrics-daemon-7d9w7","openshift-network-diagnostics/network-check-target-cxfjn","openshift-network-operator/iptables-alerter-4j47l","openshift-ovn-kubernetes/ovnkube-node-kq5w4","kube-system/global-pull-secret-syncer-525n9","kube-system/konnectivity-agent-cdhlp","kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal","openshift-cluster-node-tuning-operator/tuned-4fcgn","openshift-image-registry/node-ca-5n94l","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal","openshift-multus/multus-98wvv"] Jul 18 01:48:12.426596 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.426571 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.428897 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.428874 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.428992 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.428909 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 18 01:48:12.428992 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.428949 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 18 01:48:12.429202 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.429186 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 18 01:48:12.429260 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.429212 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-p7cgk\"" Jul 18 01:48:12.429260 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.429229 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 18 01:48:12.429355 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.429296 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 18 01:48:12.429913 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.429898 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 18 01:48:12.431001 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.430985 2548 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 18 01:48:12.431602 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.431589 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.431951 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.431932 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 18 01:48:12.432055 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.432036 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 18 01:48:12.432149 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.432134 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pl4mr\"" Jul 18 01:48:12.433811 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.433794 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 18 01:48:12.433880 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.433797 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fnm9l\"" Jul 18 01:48:12.433880 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.433852 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 18 01:48:12.433974 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.433853 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 18 01:48:12.434100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.434085 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 18 01:48:12.434164 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.434087 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 18 01:48:12.434435 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.434418 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:12.434539 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:12.434508 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:12.436858 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.436839 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:12.436943 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.436922 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdvxp\" (UniqueName: \"kubernetes.io/projected/fea0f11b-353d-4fe1-86bb-08286c333fd3-kube-api-access-mdvxp\") pod \"network-metrics-daemon-7d9w7\" (UID: \"fea0f11b-353d-4fe1-86bb-08286c333fd3\") " pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:12.436983 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.436960 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-run-ovn\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.437015 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.436984 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-cni-netd\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.437015 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:12.436980 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:12.437015 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437000 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb8b4e1e-302b-4634-a387-8569ab5be169-ovnkube-config\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.437111 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437036 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb8b4e1e-302b-4634-a387-8569ab5be169-env-overrides\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.437111 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437069 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-systemd\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.437111 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437093 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8061f1d-195f-4a14-a518-5b0b0275471e-cnibin\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.437229 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437126 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs\") pod \"network-metrics-daemon-7d9w7\" (UID: \"fea0f11b-353d-4fe1-86bb-08286c333fd3\") " pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:12.437229 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437157 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-kubernetes\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.437229 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437173 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-kubelet\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.437339 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437222 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-slash\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.437339 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437263 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-lib-modules\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.437339 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437316 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-tuned\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.437440 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437349 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8061f1d-195f-4a14-a518-5b0b0275471e-system-cni-dir\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.437440 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437382 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a8061f1d-195f-4a14-a518-5b0b0275471e-os-release\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.437440 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437419 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a8061f1d-195f-4a14-a518-5b0b0275471e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.437605 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437439 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-run-netns\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.437605 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437455 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-var-lib-openvswitch\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.437605 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437493 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-etc-openvswitch\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.437605 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437524 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-sysctl-d\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.437605 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437548 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-sysctl-conf\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.437605 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437569 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-host\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.437605 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437597 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/342f7f5d-cf7a-4256-9525-95f7db41efbb-tmp\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.437849 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437624 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-systemd-units\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.437849 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437655 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-run-openvswitch\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.437849 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437684 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-node-log\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.437849 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437707 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-sysconfig\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.437849 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437737 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-sys\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.437849 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437794 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-var-lib-kubelet\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.437849 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437819 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8061f1d-195f-4a14-a518-5b0b0275471e-cni-binary-copy\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.437849 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437840 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-cni-bin\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.438129 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437854 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb8b4e1e-302b-4634-a387-8569ab5be169-ovn-node-metrics-cert\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.438129 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437869 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb8b4e1e-302b-4634-a387-8569ab5be169-ovnkube-script-lib\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.438129 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437883 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-modprobe-d\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.438129 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437896 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-run\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.438129 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437911 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv9d4\" (UniqueName: \"kubernetes.io/projected/a8061f1d-195f-4a14-a518-5b0b0275471e-kube-api-access-mv9d4\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.438129 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437928 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.438129 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437944 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klftj\" (UniqueName: \"kubernetes.io/projected/cb8b4e1e-302b-4634-a387-8569ab5be169-kube-api-access-klftj\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.438129 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.437961 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5jv4\" (UniqueName: \"kubernetes.io/projected/342f7f5d-cf7a-4256-9525-95f7db41efbb-kube-api-access-v5jv4\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.438129 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.438001 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a8061f1d-195f-4a14-a518-5b0b0275471e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.438129 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.438046 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-run-systemd\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.438129 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.438067 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-log-socket\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.438129 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.438088 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-run-ovn-kubernetes\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.438129 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.438128 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8061f1d-195f-4a14-a518-5b0b0275471e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.439644 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.439628 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.442109 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.442092 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cdhlp" Jul 18 01:48:12.442463 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.442435 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 18 01:48:12.442538 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.442529 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-h7j7q\"" Jul 18 01:48:12.443930 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.443908 2548 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 18 01:48:12.444535 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.444522 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zxhpn\"" Jul 18 01:48:12.444826 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.444811 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 18 01:48:12.444892 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.444850 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 18 01:48:12.445344 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.445328 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:12.445400 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:12.445386 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:12.447764 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.447752 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bwx4x" Jul 18 01:48:12.449958 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.449943 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 18 01:48:12.450277 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.450237 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5n94l" Jul 18 01:48:12.451030 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.450959 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-r4d6w\"" Jul 18 01:48:12.452746 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.451326 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 18 01:48:12.453188 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.453152 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-qffkh\"" Jul 18 01:48:12.453286 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.453216 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 18 01:48:12.453286 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.453211 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 18 01:48:12.453406 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.453306 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 18 01:48:12.453664 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.453644 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.455915 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.455898 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 18 01:48:12.456172 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.456156 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dmbfz\"" Jul 18 01:48:12.456228 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.456168 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 18 01:48:12.456272 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.456239 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 18 01:48:12.456584 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.456566 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4j47l" Jul 18 01:48:12.458668 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.458650 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 18 01:48:12.458761 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.458694 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 18 01:48:12.458761 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.458697 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 18 01:48:12.458895 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.458699 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-llt6v\"" Jul 18 01:48:12.463209 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.463161 2548 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-17 01:43:11 +0000 UTC" deadline="2028-03-06 06:11:38.948970304 +0000 UTC" Jul 18 01:48:12.463209 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.463207 2548 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14332h23m26.485765264s" Jul 18 01:48:12.476070 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.476050 2548 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-q8ch9" Jul 18 01:48:12.486634 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.486613 2548 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-q8ch9" Jul 18 01:48:12.532637 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.532620 2548 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 18 01:48:12.538354 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538334 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-os-release\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.538458 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538365 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-systemd-units\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.538458 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538382 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb8b4e1e-302b-4634-a387-8569ab5be169-ovnkube-script-lib\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.538458 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538398 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-var-lib-kubelet\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.538639 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538454 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6d4e467b-f727-41ee-94b4-eb64e672a7dc-konnectivity-ca\") pod \"konnectivity-agent-cdhlp\" (UID: \"6d4e467b-f727-41ee-94b4-eb64e672a7dc\") " pod="kube-system/konnectivity-agent-cdhlp" Jul 18 01:48:12.538639 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538469 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-systemd-units\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.538639 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538494 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-sys\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.538639 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538521 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-var-lib-kubelet\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.538639 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538546 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-cni-bin\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.538639 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538571 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb8b4e1e-302b-4634-a387-8569ab5be169-ovn-node-metrics-cert\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.538639 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538597 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/79e51bb5-b3d5-4003-8c70-f789f06b54e8-kubelet-config\") pod \"global-pull-secret-syncer-525n9\" (UID: \"79e51bb5-b3d5-4003-8c70-f789f06b54e8\") " pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:12.538639 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538613 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-sys\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.538639 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538618 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-var-lib-kubelet\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.538639 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538626 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-cni-bin\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.538639 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538625 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.539254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538675 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5jv4\" (UniqueName: \"kubernetes.io/projected/342f7f5d-cf7a-4256-9525-95f7db41efbb-kube-api-access-v5jv4\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.539254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538684 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.539254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538695 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a8061f1d-195f-4a14-a518-5b0b0275471e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.539254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538713 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-log-socket\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.539254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538751 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2ded5998-7e45-4049-9302-bab9411039ba-serviceca\") pod \"node-ca-5n94l\" (UID: \"2ded5998-7e45-4049-9302-bab9411039ba\") " pod="openshift-image-registry/node-ca-5n94l" Jul 18 01:48:12.539254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538771 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-log-socket\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.539254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538791 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97l7b\" (UniqueName: \"kubernetes.io/projected/c8f58c36-699d-4215-9b30-98e166a03995-kube-api-access-97l7b\") pod \"iptables-alerter-4j47l\" (UID: \"c8f58c36-699d-4215-9b30-98e166a03995\") " pod="openshift-network-operator/iptables-alerter-4j47l" Jul 18 01:48:12.539254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538815 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b-tmp-dir\") pod \"node-resolver-bwx4x\" (UID: \"2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b\") " pod="openshift-dns/node-resolver-bwx4x" Jul 18 01:48:12.539254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538834 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8061f1d-195f-4a14-a518-5b0b0275471e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.539254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538849 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-run-ovn\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.539254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538869 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-cni-netd\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.539254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538895 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b-hosts-file\") pod \"node-resolver-bwx4x\" (UID: \"2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b\") " pod="openshift-dns/node-resolver-bwx4x" Jul 18 01:48:12.539254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538917 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/79e51bb5-b3d5-4003-8c70-f789f06b54e8-dbus\") pod \"global-pull-secret-syncer-525n9\" (UID: \"79e51bb5-b3d5-4003-8c70-f789f06b54e8\") " pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:12.539254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538920 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-cni-netd\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.539254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538921 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-run-ovn\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.539254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538955 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8061f1d-195f-4a14-a518-5b0b0275471e-cnibin\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.539254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538978 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs\") pod \"network-metrics-daemon-7d9w7\" (UID: \"fea0f11b-353d-4fe1-86bb-08286c333fd3\") " pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.538996 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crs2w\" (UniqueName: \"kubernetes.io/projected/2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b-kube-api-access-crs2w\") pod \"node-resolver-bwx4x\" (UID: \"2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b\") " pod="openshift-dns/node-resolver-bwx4x" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539013 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-registration-dir\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539035 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-multus-daemon-config\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539059 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-kubernetes\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539064 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a8061f1d-195f-4a14-a518-5b0b0275471e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539082 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-kubelet\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539093 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cb8b4e1e-302b-4634-a387-8569ab5be169-ovnkube-script-lib\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539104 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72f4q\" (UniqueName: \"kubernetes.io/projected/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-kube-api-access-72f4q\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539106 2548 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539118 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a8061f1d-195f-4a14-a518-5b0b0275471e-cnibin\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539134 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-run-netns\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539161 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-kubelet\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539162 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6d4e467b-f727-41ee-94b4-eb64e672a7dc-agent-certs\") pod \"konnectivity-agent-cdhlp\" (UID: \"6d4e467b-f727-41ee-94b4-eb64e672a7dc\") " pod="kube-system/konnectivity-agent-cdhlp" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539165 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-kubernetes\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539197 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-lib-modules\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539222 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-tuned\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539299 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8061f1d-195f-4a14-a518-5b0b0275471e-system-cni-dir\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.540196 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:12.539098 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:12.541094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539368 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a8061f1d-195f-4a14-a518-5b0b0275471e-system-cni-dir\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.541094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539373 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-lib-modules\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.541094 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:12.539457 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs podName:fea0f11b-353d-4fe1-86bb-08286c333fd3 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:13.039423857 +0000 UTC m=+1.892167183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs") pod "network-metrics-daemon-7d9w7" (UID: "fea0f11b-353d-4fe1-86bb-08286c333fd3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:12.541094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539521 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a8061f1d-195f-4a14-a518-5b0b0275471e-os-release\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.541094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539612 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a8061f1d-195f-4a14-a518-5b0b0275471e-os-release\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.541094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539652 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a8061f1d-195f-4a14-a518-5b0b0275471e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.541094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539679 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-var-lib-openvswitch\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.541094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539700 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-run-openvswitch\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.541094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539720 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-node-log\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.541094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539740 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-sysctl-d\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.541094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539760 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-sysctl-conf\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.541094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539797 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-host\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.541094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539817 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/342f7f5d-cf7a-4256-9525-95f7db41efbb-tmp\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.541094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539847 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-run-openvswitch\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.541094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539847 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a8061f1d-195f-4a14-a518-5b0b0275471e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.541094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539854 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvgk2\" (UniqueName: \"kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2\") pod \"network-check-target-cxfjn\" (UID: \"2939a38f-6e71-4bfa-86ea-82969887427d\") " pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:12.541886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539940 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-host\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.541886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539956 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-sysctl-conf\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.541886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539973 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-var-lib-openvswitch\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.541886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539995 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-sysctl-d\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.541886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.539998 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.541886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540004 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-node-log\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.541886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540025 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-device-dir\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.541886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540048 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-sysconfig\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.541886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540087 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8061f1d-195f-4a14-a518-5b0b0275471e-cni-binary-copy\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.541886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540112 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-sysconfig\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.541886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540151 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-socket-dir\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.541886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540180 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-sys-fs\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.541886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540217 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-system-cni-dir\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.541886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540248 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-multus-socket-dir-parent\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.541886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540273 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-var-lib-cni-bin\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.541886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540312 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-hostroot\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.541886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540341 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-modprobe-d\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.542657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540377 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-run\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.542657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540403 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv9d4\" (UniqueName: \"kubernetes.io/projected/a8061f1d-195f-4a14-a518-5b0b0275471e-kube-api-access-mv9d4\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.542657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540453 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klftj\" (UniqueName: \"kubernetes.io/projected/cb8b4e1e-302b-4634-a387-8569ab5be169-kube-api-access-klftj\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.542657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540478 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.542657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540519 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-multus-cni-dir\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.542657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540544 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-cni-binary-copy\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.542657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540614 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-run-k8s-cni-cncf-io\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.542657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540643 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-run-systemd\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.542657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540649 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a8061f1d-195f-4a14-a518-5b0b0275471e-cni-binary-copy\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.542657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540666 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-run-ovn-kubernetes\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.542657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540690 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8f58c36-699d-4215-9b30-98e166a03995-host-slash\") pod \"iptables-alerter-4j47l\" (UID: \"c8f58c36-699d-4215-9b30-98e166a03995\") " pod="openshift-network-operator/iptables-alerter-4j47l" Jul 18 01:48:12.542657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540713 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-cnibin\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.542657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540757 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-var-lib-cni-multus\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.542657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540821 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-run-ovn-kubernetes\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.542657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540906 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-modprobe-d\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.542657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540939 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-run\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.542657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541048 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a8061f1d-195f-4a14-a518-5b0b0275471e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541180 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-etc-kubernetes\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.540768 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-run-systemd\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541211 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhvrm\" (UniqueName: \"kubernetes.io/projected/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-kube-api-access-qhvrm\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541246 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdvxp\" (UniqueName: \"kubernetes.io/projected/fea0f11b-353d-4fe1-86bb-08286c333fd3-kube-api-access-mdvxp\") pod \"network-metrics-daemon-7d9w7\" (UID: \"fea0f11b-353d-4fe1-86bb-08286c333fd3\") " pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:12.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541326 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb8b4e1e-302b-4634-a387-8569ab5be169-ovnkube-config\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541397 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb8b4e1e-302b-4634-a387-8569ab5be169-env-overrides\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541422 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5nhs\" (UniqueName: \"kubernetes.io/projected/2ded5998-7e45-4049-9302-bab9411039ba-kube-api-access-s5nhs\") pod \"node-ca-5n94l\" (UID: \"2ded5998-7e45-4049-9302-bab9411039ba\") " pod="openshift-image-registry/node-ca-5n94l" Jul 18 01:48:12.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541457 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret\") pod \"global-pull-secret-syncer-525n9\" (UID: \"79e51bb5-b3d5-4003-8c70-f789f06b54e8\") " pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:12.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541482 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-systemd\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541505 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ded5998-7e45-4049-9302-bab9411039ba-host\") pod \"node-ca-5n94l\" (UID: \"2ded5998-7e45-4049-9302-bab9411039ba\") " pod="openshift-image-registry/node-ca-5n94l" Jul 18 01:48:12.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541529 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c8f58c36-699d-4215-9b30-98e166a03995-iptables-alerter-script\") pod \"iptables-alerter-4j47l\" (UID: \"c8f58c36-699d-4215-9b30-98e166a03995\") " pod="openshift-network-operator/iptables-alerter-4j47l" Jul 18 01:48:12.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541552 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-multus-conf-dir\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541577 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-slash\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541600 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-run-multus-certs\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541624 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-run-netns\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541648 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-etc-openvswitch\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541732 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-etc-openvswitch\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.543929 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541817 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-systemd\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.543929 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541820 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cb8b4e1e-302b-4634-a387-8569ab5be169-env-overrides\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.543929 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541827 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cb8b4e1e-302b-4634-a387-8569ab5be169-ovnkube-config\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.543929 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541864 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-slash\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.543929 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.541901 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cb8b4e1e-302b-4634-a387-8569ab5be169-host-run-netns\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.543929 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.542715 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/342f7f5d-cf7a-4256-9525-95f7db41efbb-etc-tuned\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.543929 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.542945 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cb8b4e1e-302b-4634-a387-8569ab5be169-ovn-node-metrics-cert\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.543929 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.543429 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/342f7f5d-cf7a-4256-9525-95f7db41efbb-tmp\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.546420 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.546400 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5jv4\" (UniqueName: \"kubernetes.io/projected/342f7f5d-cf7a-4256-9525-95f7db41efbb-kube-api-access-v5jv4\") pod \"tuned-4fcgn\" (UID: \"342f7f5d-cf7a-4256-9525-95f7db41efbb\") " pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.550281 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.550234 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdvxp\" (UniqueName: \"kubernetes.io/projected/fea0f11b-353d-4fe1-86bb-08286c333fd3-kube-api-access-mdvxp\") pod \"network-metrics-daemon-7d9w7\" (UID: \"fea0f11b-353d-4fe1-86bb-08286c333fd3\") " pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:12.551296 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.551253 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv9d4\" (UniqueName: \"kubernetes.io/projected/a8061f1d-195f-4a14-a518-5b0b0275471e-kube-api-access-mv9d4\") pod \"multus-additional-cni-plugins-hvmsl\" (UID: \"a8061f1d-195f-4a14-a518-5b0b0275471e\") " pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.551296 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.551285 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klftj\" (UniqueName: \"kubernetes.io/projected/cb8b4e1e-302b-4634-a387-8569ab5be169-kube-api-access-klftj\") pod \"ovnkube-node-kq5w4\" (UID: \"cb8b4e1e-302b-4634-a387-8569ab5be169\") " pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.600181 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:12.600155 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ebbea173dadad9136c4ee6a95a9762e.slice/crio-0754cf913dd468c5148ab7fb9f17eb55f4c1e1c304a0ddca70f00acfe664335e WatchSource:0}: Error finding container 0754cf913dd468c5148ab7fb9f17eb55f4c1e1c304a0ddca70f00acfe664335e: Status 404 returned error can't find the container with id 0754cf913dd468c5148ab7fb9f17eb55f4c1e1c304a0ddca70f00acfe664335e Jul 18 01:48:12.600689 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:12.600667 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d448a47aa0f5878864b37d16dc8c112.slice/crio-6d9999cbef87858fff9acc4642ff0f7a7b1617bf0e1ee77b2e4d6e1165d1bf4c WatchSource:0}: Error finding container 6d9999cbef87858fff9acc4642ff0f7a7b1617bf0e1ee77b2e4d6e1165d1bf4c: Status 404 returned error can't find the container with id 6d9999cbef87858fff9acc4642ff0f7a7b1617bf0e1ee77b2e4d6e1165d1bf4c Jul 18 01:48:12.605564 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.605550 2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 01:48:12.642217 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642191 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-var-lib-kubelet\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.642295 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642223 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6d4e467b-f727-41ee-94b4-eb64e672a7dc-konnectivity-ca\") pod \"konnectivity-agent-cdhlp\" (UID: \"6d4e467b-f727-41ee-94b4-eb64e672a7dc\") " pod="kube-system/konnectivity-agent-cdhlp" Jul 18 01:48:12.642295 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642241 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/79e51bb5-b3d5-4003-8c70-f789f06b54e8-kubelet-config\") pod \"global-pull-secret-syncer-525n9\" (UID: \"79e51bb5-b3d5-4003-8c70-f789f06b54e8\") " pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:12.642295 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642265 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2ded5998-7e45-4049-9302-bab9411039ba-serviceca\") pod \"node-ca-5n94l\" (UID: \"2ded5998-7e45-4049-9302-bab9411039ba\") " pod="openshift-image-registry/node-ca-5n94l" Jul 18 01:48:12.642399 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642296 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-var-lib-kubelet\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.642399 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642333 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/79e51bb5-b3d5-4003-8c70-f789f06b54e8-kubelet-config\") pod \"global-pull-secret-syncer-525n9\" (UID: \"79e51bb5-b3d5-4003-8c70-f789f06b54e8\") " pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:12.642399 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642370 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97l7b\" (UniqueName: \"kubernetes.io/projected/c8f58c36-699d-4215-9b30-98e166a03995-kube-api-access-97l7b\") pod \"iptables-alerter-4j47l\" (UID: \"c8f58c36-699d-4215-9b30-98e166a03995\") " pod="openshift-network-operator/iptables-alerter-4j47l" Jul 18 01:48:12.642399 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642389 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b-tmp-dir\") pod \"node-resolver-bwx4x\" (UID: \"2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b\") " pod="openshift-dns/node-resolver-bwx4x" Jul 18 01:48:12.642572 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642410 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b-hosts-file\") pod \"node-resolver-bwx4x\" (UID: \"2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b\") " pod="openshift-dns/node-resolver-bwx4x" Jul 18 01:48:12.642572 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642432 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/79e51bb5-b3d5-4003-8c70-f789f06b54e8-dbus\") pod \"global-pull-secret-syncer-525n9\" (UID: \"79e51bb5-b3d5-4003-8c70-f789f06b54e8\") " pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:12.642572 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642477 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crs2w\" (UniqueName: \"kubernetes.io/projected/2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b-kube-api-access-crs2w\") pod \"node-resolver-bwx4x\" (UID: \"2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b\") " pod="openshift-dns/node-resolver-bwx4x" Jul 18 01:48:12.642572 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642525 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-registration-dir\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.642572 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642537 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b-hosts-file\") pod \"node-resolver-bwx4x\" (UID: \"2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b\") " pod="openshift-dns/node-resolver-bwx4x" Jul 18 01:48:12.642572 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642557 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-multus-daemon-config\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.642837 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642587 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72f4q\" (UniqueName: \"kubernetes.io/projected/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-kube-api-access-72f4q\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.642837 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642591 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/79e51bb5-b3d5-4003-8c70-f789f06b54e8-dbus\") pod \"global-pull-secret-syncer-525n9\" (UID: \"79e51bb5-b3d5-4003-8c70-f789f06b54e8\") " pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:12.642837 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642605 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-registration-dir\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.642837 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642613 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-run-netns\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.642837 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642636 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6d4e467b-f727-41ee-94b4-eb64e672a7dc-agent-certs\") pod \"konnectivity-agent-cdhlp\" (UID: \"6d4e467b-f727-41ee-94b4-eb64e672a7dc\") " pod="kube-system/konnectivity-agent-cdhlp" Jul 18 01:48:12.642837 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642671 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvgk2\" (UniqueName: \"kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2\") pod \"network-check-target-cxfjn\" (UID: \"2939a38f-6e71-4bfa-86ea-82969887427d\") " pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:12.642837 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642692 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.642837 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642714 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-device-dir\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.642837 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642736 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-socket-dir\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.642837 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642759 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-sys-fs\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.642837 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642768 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-run-netns\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.642837 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642794 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6d4e467b-f727-41ee-94b4-eb64e672a7dc-konnectivity-ca\") pod \"konnectivity-agent-cdhlp\" (UID: \"6d4e467b-f727-41ee-94b4-eb64e672a7dc\") " pod="kube-system/konnectivity-agent-cdhlp" Jul 18 01:48:12.642837 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642800 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-system-cni-dir\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.642837 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642813 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-device-dir\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.642837 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642822 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.642837 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642840 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-multus-socket-dir-parent\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642847 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-system-cni-dir\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642868 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-var-lib-cni-bin\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642892 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-hostroot\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642895 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-sys-fs\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642902 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-multus-socket-dir-parent\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642921 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642923 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-socket-dir\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642947 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-var-lib-cni-bin\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642965 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-hostroot\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.642976 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-multus-cni-dir\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643002 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-cni-binary-copy\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643006 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-etc-selinux\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643022 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-multus-cni-dir\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643037 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-run-k8s-cni-cncf-io\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643070 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8f58c36-699d-4215-9b30-98e166a03995-host-slash\") pod \"iptables-alerter-4j47l\" (UID: \"c8f58c36-699d-4215-9b30-98e166a03995\") " pod="openshift-network-operator/iptables-alerter-4j47l" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643093 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-cnibin\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643117 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-var-lib-cni-multus\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.643576 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643128 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-multus-daemon-config\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643133 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-run-k8s-cni-cncf-io\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643140 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-etc-kubernetes\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643165 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhvrm\" (UniqueName: \"kubernetes.io/projected/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-kube-api-access-qhvrm\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643189 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-etc-kubernetes\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643192 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5nhs\" (UniqueName: \"kubernetes.io/projected/2ded5998-7e45-4049-9302-bab9411039ba-kube-api-access-s5nhs\") pod \"node-ca-5n94l\" (UID: \"2ded5998-7e45-4049-9302-bab9411039ba\") " pod="openshift-image-registry/node-ca-5n94l" Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643217 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret\") pod \"global-pull-secret-syncer-525n9\" (UID: \"79e51bb5-b3d5-4003-8c70-f789f06b54e8\") " pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643245 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ded5998-7e45-4049-9302-bab9411039ba-host\") pod \"node-ca-5n94l\" (UID: \"2ded5998-7e45-4049-9302-bab9411039ba\") " pod="openshift-image-registry/node-ca-5n94l" Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643248 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-cnibin\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643257 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c8f58c36-699d-4215-9b30-98e166a03995-host-slash\") pod \"iptables-alerter-4j47l\" (UID: \"c8f58c36-699d-4215-9b30-98e166a03995\") " pod="openshift-network-operator/iptables-alerter-4j47l" Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643267 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c8f58c36-699d-4215-9b30-98e166a03995-iptables-alerter-script\") pod \"iptables-alerter-4j47l\" (UID: \"c8f58c36-699d-4215-9b30-98e166a03995\") " pod="openshift-network-operator/iptables-alerter-4j47l" Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643332 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-multus-conf-dir\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643365 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-run-multus-certs\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643377 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2ded5998-7e45-4049-9302-bab9411039ba-serviceca\") pod \"node-ca-5n94l\" (UID: \"2ded5998-7e45-4049-9302-bab9411039ba\") " pod="openshift-image-registry/node-ca-5n94l" Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643392 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-os-release\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:12.643435 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643454 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-cni-binary-copy\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643466 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-os-release\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.644191 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:12.643494 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret podName:79e51bb5-b3d5-4003-8c70-f789f06b54e8 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:13.143476677 +0000 UTC m=+1.996220016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret") pod "global-pull-secret-syncer-525n9" (UID: "79e51bb5-b3d5-4003-8c70-f789f06b54e8") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:12.644688 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643513 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-var-lib-cni-multus\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.644688 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643553 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-host-run-multus-certs\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.644688 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643589 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-multus-conf-dir\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.644688 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643623 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2ded5998-7e45-4049-9302-bab9411039ba-host\") pod \"node-ca-5n94l\" (UID: \"2ded5998-7e45-4049-9302-bab9411039ba\") " pod="openshift-image-registry/node-ca-5n94l" Jul 18 01:48:12.644688 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643818 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b-tmp-dir\") pod \"node-resolver-bwx4x\" (UID: \"2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b\") " pod="openshift-dns/node-resolver-bwx4x" Jul 18 01:48:12.644688 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.643853 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c8f58c36-699d-4215-9b30-98e166a03995-iptables-alerter-script\") pod \"iptables-alerter-4j47l\" (UID: \"c8f58c36-699d-4215-9b30-98e166a03995\") " pod="openshift-network-operator/iptables-alerter-4j47l" Jul 18 01:48:12.645075 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.645059 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6d4e467b-f727-41ee-94b4-eb64e672a7dc-agent-certs\") pod \"konnectivity-agent-cdhlp\" (UID: \"6d4e467b-f727-41ee-94b4-eb64e672a7dc\") " pod="kube-system/konnectivity-agent-cdhlp" Jul 18 01:48:12.650885 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:12.650864 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:48:12.650985 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:12.650890 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:48:12.650985 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:12.650903 2548 projected.go:196] Error preparing data for projected volume kube-api-access-mvgk2 for pod openshift-network-diagnostics/network-check-target-cxfjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:12.650985 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:12.650960 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2 podName:2939a38f-6e71-4bfa-86ea-82969887427d nodeName:}" failed. No retries permitted until 2026-07-18 01:48:13.150942093 +0000 UTC m=+2.003685432 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mvgk2" (UniqueName: "kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2") pod "network-check-target-cxfjn" (UID: "2939a38f-6e71-4bfa-86ea-82969887427d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:12.651314 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.651296 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crs2w\" (UniqueName: \"kubernetes.io/projected/2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b-kube-api-access-crs2w\") pod \"node-resolver-bwx4x\" (UID: \"2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b\") " pod="openshift-dns/node-resolver-bwx4x" Jul 18 01:48:12.651549 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.651525 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97l7b\" (UniqueName: \"kubernetes.io/projected/c8f58c36-699d-4215-9b30-98e166a03995-kube-api-access-97l7b\") pod \"iptables-alerter-4j47l\" (UID: \"c8f58c36-699d-4215-9b30-98e166a03995\") " pod="openshift-network-operator/iptables-alerter-4j47l" Jul 18 01:48:12.652056 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.652041 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5nhs\" (UniqueName: \"kubernetes.io/projected/2ded5998-7e45-4049-9302-bab9411039ba-kube-api-access-s5nhs\") pod \"node-ca-5n94l\" (UID: \"2ded5998-7e45-4049-9302-bab9411039ba\") " pod="openshift-image-registry/node-ca-5n94l" Jul 18 01:48:12.652241 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.652225 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72f4q\" (UniqueName: \"kubernetes.io/projected/43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0-kube-api-access-72f4q\") pod \"aws-ebs-csi-driver-node-jcvl9\" (UID: \"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.652805 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.652789 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhvrm\" (UniqueName: \"kubernetes.io/projected/33094a8a-fe46-41a7-ba7a-cdf29c47e2d7-kube-api-access-qhvrm\") pod \"multus-98wvv\" (UID: \"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7\") " pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.715586 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.715568 2548 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:48:12.758728 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.758678 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:12.768327 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:12.768303 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb8b4e1e_302b_4634_a387_8569ab5be169.slice/crio-1a273ae19d0898723cffe4b6a3daa23579213ed1db87ce68a3666bb9828d0501 WatchSource:0}: Error finding container 1a273ae19d0898723cffe4b6a3daa23579213ed1db87ce68a3666bb9828d0501: Status 404 returned error can't find the container with id 1a273ae19d0898723cffe4b6a3daa23579213ed1db87ce68a3666bb9828d0501 Jul 18 01:48:12.772323 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.772308 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" Jul 18 01:48:12.777232 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.777214 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hvmsl" Jul 18 01:48:12.780031 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:12.780014 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod342f7f5d_cf7a_4256_9525_95f7db41efbb.slice/crio-479b31b89ceed9e554cbe97261f317a399eea7ca5e9a465ef3468d008eeb7ef8 WatchSource:0}: Error finding container 479b31b89ceed9e554cbe97261f317a399eea7ca5e9a465ef3468d008eeb7ef8: Status 404 returned error can't find the container with id 479b31b89ceed9e554cbe97261f317a399eea7ca5e9a465ef3468d008eeb7ef8 Jul 18 01:48:12.785728 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:12.785687 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8061f1d_195f_4a14_a518_5b0b0275471e.slice/crio-9ea8ba37695aa7546048c3bd6271fba7b960c0709f5f819c126b0e5570e01f9e WatchSource:0}: Error finding container 9ea8ba37695aa7546048c3bd6271fba7b960c0709f5f819c126b0e5570e01f9e: Status 404 returned error can't find the container with id 9ea8ba37695aa7546048c3bd6271fba7b960c0709f5f819c126b0e5570e01f9e Jul 18 01:48:12.793468 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.793453 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-98wvv" Jul 18 01:48:12.801399 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:12.801381 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33094a8a_fe46_41a7_ba7a_cdf29c47e2d7.slice/crio-8d3e72c02ba46e3dc91f60e3955a5b4821eab3266709fe1154ea92d025482e47 WatchSource:0}: Error finding container 8d3e72c02ba46e3dc91f60e3955a5b4821eab3266709fe1154ea92d025482e47: Status 404 returned error can't find the container with id 8d3e72c02ba46e3dc91f60e3955a5b4821eab3266709fe1154ea92d025482e47 Jul 18 01:48:12.802687 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.802676 2548 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:48:12.808635 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.808620 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cdhlp" Jul 18 01:48:12.815915 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:12.815893 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d4e467b_f727_41ee_94b4_eb64e672a7dc.slice/crio-1e0f77e3ed97d1a7e0a70e1345e3672672e6014eee12d0ef4659bcb441c0b221 WatchSource:0}: Error finding container 1e0f77e3ed97d1a7e0a70e1345e3672672e6014eee12d0ef4659bcb441c0b221: Status 404 returned error can't find the container with id 1e0f77e3ed97d1a7e0a70e1345e3672672e6014eee12d0ef4659bcb441c0b221 Jul 18 01:48:12.821544 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.821524 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bwx4x" Jul 18 01:48:12.829014 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:12.828989 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f1332dd_5e85_4bdf_b87a_ae1c5f147c3b.slice/crio-624fa465c2c56957cb1f54fd589e68a1df1d0e8ef93cdf60905975866df11339 WatchSource:0}: Error finding container 624fa465c2c56957cb1f54fd589e68a1df1d0e8ef93cdf60905975866df11339: Status 404 returned error can't find the container with id 624fa465c2c56957cb1f54fd589e68a1df1d0e8ef93cdf60905975866df11339 Jul 18 01:48:12.841798 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.841761 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5n94l" Jul 18 01:48:12.849005 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:12.848987 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ded5998_7e45_4049_9302_bab9411039ba.slice/crio-c106ac37144ceade1d97e65d90d44ec740652f85bb0b4f0a4b07df56e01ebb3d WatchSource:0}: Error finding container c106ac37144ceade1d97e65d90d44ec740652f85bb0b4f0a4b07df56e01ebb3d: Status 404 returned error can't find the container with id c106ac37144ceade1d97e65d90d44ec740652f85bb0b4f0a4b07df56e01ebb3d Jul 18 01:48:12.863145 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.863132 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" Jul 18 01:48:12.868661 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:12.868641 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4j47l" Jul 18 01:48:12.871597 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:12.871578 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43fed8c5_94c5_4b8f_ac7c_91e2f4e3a1a0.slice/crio-6e3bb1031d257f07a77e50d8555f91da1cf7f67a28fe1ba85f7fd7d311eda291 WatchSource:0}: Error finding container 6e3bb1031d257f07a77e50d8555f91da1cf7f67a28fe1ba85f7fd7d311eda291: Status 404 returned error can't find the container with id 6e3bb1031d257f07a77e50d8555f91da1cf7f67a28fe1ba85f7fd7d311eda291 Jul 18 01:48:12.876098 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:12.876080 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8f58c36_699d_4215_9b30_98e166a03995.slice/crio-47437113c6a3d38e770fcbfb59dabad01b402623809451da710bcbbb289e1367 WatchSource:0}: Error finding container 47437113c6a3d38e770fcbfb59dabad01b402623809451da710bcbbb289e1367: Status 404 returned error can't find the container with id 47437113c6a3d38e770fcbfb59dabad01b402623809451da710bcbbb289e1367 Jul 18 01:48:13.045791 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:13.045700 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs\") pod \"network-metrics-daemon-7d9w7\" (UID: \"fea0f11b-353d-4fe1-86bb-08286c333fd3\") " pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:13.045908 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:13.045842 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:13.045908 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:13.045894 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs podName:fea0f11b-353d-4fe1-86bb-08286c333fd3 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:14.045880251 +0000 UTC m=+2.898623573 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs") pod "network-metrics-daemon-7d9w7" (UID: "fea0f11b-353d-4fe1-86bb-08286c333fd3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:13.147030 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:13.146921 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret\") pod \"global-pull-secret-syncer-525n9\" (UID: \"79e51bb5-b3d5-4003-8c70-f789f06b54e8\") " pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:13.147170 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:13.147049 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:13.147170 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:13.147117 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret podName:79e51bb5-b3d5-4003-8c70-f789f06b54e8 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:14.147098332 +0000 UTC m=+2.999841672 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret") pod "global-pull-secret-syncer-525n9" (UID: "79e51bb5-b3d5-4003-8c70-f789f06b54e8") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:13.248266 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:13.248215 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvgk2\" (UniqueName: \"kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2\") pod \"network-check-target-cxfjn\" (UID: \"2939a38f-6e71-4bfa-86ea-82969887427d\") " pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:13.248426 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:13.248413 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:48:13.248491 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:13.248436 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:48:13.248491 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:13.248450 2548 projected.go:196] Error preparing data for projected volume kube-api-access-mvgk2 for pod openshift-network-diagnostics/network-check-target-cxfjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:13.248594 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:13.248510 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2 podName:2939a38f-6e71-4bfa-86ea-82969887427d nodeName:}" failed. No retries permitted until 2026-07-18 01:48:14.248492046 +0000 UTC m=+3.101235383 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mvgk2" (UniqueName: "kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2") pod "network-check-target-cxfjn" (UID: "2939a38f-6e71-4bfa-86ea-82969887427d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:13.416167 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:13.416088 2548 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 18 01:48:13.490123 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:13.489978 2548 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:43:12 +0000 UTC" deadline="2028-04-30 21:02:49.739497252 +0000 UTC" Jul 18 01:48:13.490123 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:13.490038 2548 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15667h14m36.249462706s" Jul 18 01:48:13.596591 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:13.596536 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5n94l" event={"ID":"2ded5998-7e45-4049-9302-bab9411039ba","Type":"ContainerStarted","Data":"c106ac37144ceade1d97e65d90d44ec740652f85bb0b4f0a4b07df56e01ebb3d"} Jul 18 01:48:13.608989 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:13.608844 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bwx4x" event={"ID":"2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b","Type":"ContainerStarted","Data":"624fa465c2c56957cb1f54fd589e68a1df1d0e8ef93cdf60905975866df11339"} Jul 18 01:48:13.623932 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:13.623868 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cdhlp" event={"ID":"6d4e467b-f727-41ee-94b4-eb64e672a7dc","Type":"ContainerStarted","Data":"1e0f77e3ed97d1a7e0a70e1345e3672672e6014eee12d0ef4659bcb441c0b221"} Jul 18 01:48:13.626261 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:13.626206 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-98wvv" event={"ID":"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7","Type":"ContainerStarted","Data":"8d3e72c02ba46e3dc91f60e3955a5b4821eab3266709fe1154ea92d025482e47"} Jul 18 01:48:13.636646 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:13.636575 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvmsl" event={"ID":"a8061f1d-195f-4a14-a518-5b0b0275471e","Type":"ContainerStarted","Data":"9ea8ba37695aa7546048c3bd6271fba7b960c0709f5f819c126b0e5570e01f9e"} Jul 18 01:48:13.646399 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:13.646372 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" event={"ID":"342f7f5d-cf7a-4256-9525-95f7db41efbb","Type":"ContainerStarted","Data":"479b31b89ceed9e554cbe97261f317a399eea7ca5e9a465ef3468d008eeb7ef8"} Jul 18 01:48:13.675359 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:13.675284 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" event={"ID":"cb8b4e1e-302b-4634-a387-8569ab5be169","Type":"ContainerStarted","Data":"1a273ae19d0898723cffe4b6a3daa23579213ed1db87ce68a3666bb9828d0501"} Jul 18 01:48:13.689119 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:13.689087 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" event={"ID":"1ebbea173dadad9136c4ee6a95a9762e","Type":"ContainerStarted","Data":"0754cf913dd468c5148ab7fb9f17eb55f4c1e1c304a0ddca70f00acfe664335e"} Jul 18 01:48:13.700484 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:13.700451 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4j47l" event={"ID":"c8f58c36-699d-4215-9b30-98e166a03995","Type":"ContainerStarted","Data":"47437113c6a3d38e770fcbfb59dabad01b402623809451da710bcbbb289e1367"} Jul 18 01:48:13.719909 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:13.719869 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" event={"ID":"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0","Type":"ContainerStarted","Data":"6e3bb1031d257f07a77e50d8555f91da1cf7f67a28fe1ba85f7fd7d311eda291"} Jul 18 01:48:13.755831 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:13.755799 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" event={"ID":"3d448a47aa0f5878864b37d16dc8c112","Type":"ContainerStarted","Data":"6d9999cbef87858fff9acc4642ff0f7a7b1617bf0e1ee77b2e4d6e1165d1bf4c"} Jul 18 01:48:14.054426 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:14.054322 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs\") pod \"network-metrics-daemon-7d9w7\" (UID: \"fea0f11b-353d-4fe1-86bb-08286c333fd3\") " pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:14.054631 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:14.054507 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:14.054631 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:14.054564 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs podName:fea0f11b-353d-4fe1-86bb-08286c333fd3 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:16.05454563 +0000 UTC m=+4.907288954 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs") pod "network-metrics-daemon-7d9w7" (UID: "fea0f11b-353d-4fe1-86bb-08286c333fd3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:14.156299 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:14.155683 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret\") pod \"global-pull-secret-syncer-525n9\" (UID: \"79e51bb5-b3d5-4003-8c70-f789f06b54e8\") " pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:14.156299 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:14.155849 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:14.156299 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:14.155909 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret podName:79e51bb5-b3d5-4003-8c70-f789f06b54e8 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:16.155890224 +0000 UTC m=+5.008633550 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret") pod "global-pull-secret-syncer-525n9" (UID: "79e51bb5-b3d5-4003-8c70-f789f06b54e8") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:14.257227 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:14.256585 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvgk2\" (UniqueName: \"kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2\") pod \"network-check-target-cxfjn\" (UID: \"2939a38f-6e71-4bfa-86ea-82969887427d\") " pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:14.257227 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:14.256810 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:48:14.257227 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:14.256832 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:48:14.257227 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:14.256845 2548 projected.go:196] Error preparing data for projected volume kube-api-access-mvgk2 for pod openshift-network-diagnostics/network-check-target-cxfjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:14.257227 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:14.256903 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2 podName:2939a38f-6e71-4bfa-86ea-82969887427d nodeName:}" failed. No retries permitted until 2026-07-18 01:48:16.256882921 +0000 UTC m=+5.109626257 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mvgk2" (UniqueName: "kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2") pod "network-check-target-cxfjn" (UID: "2939a38f-6e71-4bfa-86ea-82969887427d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:14.491273 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:14.491058 2548 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:43:12 +0000 UTC" deadline="2028-01-05 04:04:54.085226698 +0000 UTC" Jul 18 01:48:14.491273 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:14.491093 2548 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12866h16m39.594138306s" Jul 18 01:48:14.542898 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:14.542866 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:14.543058 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:14.542999 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:14.543425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:14.543407 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:14.543535 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:14.543515 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:14.543595 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:14.543585 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:14.543669 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:14.543653 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:16.075813 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:16.075099 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs\") pod \"network-metrics-daemon-7d9w7\" (UID: \"fea0f11b-353d-4fe1-86bb-08286c333fd3\") " pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:16.075813 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:16.075363 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:16.075813 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:16.075427 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs podName:fea0f11b-353d-4fe1-86bb-08286c333fd3 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:20.075408172 +0000 UTC m=+8.928151515 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs") pod "network-metrics-daemon-7d9w7" (UID: "fea0f11b-353d-4fe1-86bb-08286c333fd3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:16.176394 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:16.176357 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret\") pod \"global-pull-secret-syncer-525n9\" (UID: \"79e51bb5-b3d5-4003-8c70-f789f06b54e8\") " pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:16.176560 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:16.176539 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:16.176616 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:16.176600 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret podName:79e51bb5-b3d5-4003-8c70-f789f06b54e8 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:20.176581578 +0000 UTC m=+9.029324917 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret") pod "global-pull-secret-syncer-525n9" (UID: "79e51bb5-b3d5-4003-8c70-f789f06b54e8") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:16.277454 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:16.277397 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvgk2\" (UniqueName: \"kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2\") pod \"network-check-target-cxfjn\" (UID: \"2939a38f-6e71-4bfa-86ea-82969887427d\") " pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:16.277628 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:16.277559 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:48:16.277628 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:16.277578 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:48:16.277628 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:16.277591 2548 projected.go:196] Error preparing data for projected volume kube-api-access-mvgk2 for pod openshift-network-diagnostics/network-check-target-cxfjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:16.277806 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:16.277653 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2 podName:2939a38f-6e71-4bfa-86ea-82969887427d nodeName:}" failed. No retries permitted until 2026-07-18 01:48:20.277633399 +0000 UTC m=+9.130376726 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mvgk2" (UniqueName: "kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2") pod "network-check-target-cxfjn" (UID: "2939a38f-6e71-4bfa-86ea-82969887427d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:16.542993 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:16.542927 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:16.543136 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:16.543029 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:16.543136 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:16.543068 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:16.543215 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:16.543194 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:16.543260 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:16.543247 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:16.543339 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:16.543321 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:18.543526 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:18.543485 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:18.543526 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:18.543525 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:18.544035 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:18.543623 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:18.544035 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:18.543658 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:18.544035 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:18.543722 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:18.544035 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:18.543805 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:20.108855 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:20.108252 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs\") pod \"network-metrics-daemon-7d9w7\" (UID: \"fea0f11b-353d-4fe1-86bb-08286c333fd3\") " pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:20.108855 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:20.108443 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:20.108855 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:20.108513 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs podName:fea0f11b-353d-4fe1-86bb-08286c333fd3 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:28.10849222 +0000 UTC m=+16.961235545 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs") pod "network-metrics-daemon-7d9w7" (UID: "fea0f11b-353d-4fe1-86bb-08286c333fd3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:20.209520 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:20.209437 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret\") pod \"global-pull-secret-syncer-525n9\" (UID: \"79e51bb5-b3d5-4003-8c70-f789f06b54e8\") " pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:20.209660 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:20.209612 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:20.209711 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:20.209679 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret podName:79e51bb5-b3d5-4003-8c70-f789f06b54e8 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:28.20965703 +0000 UTC m=+17.062400352 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret") pod "global-pull-secret-syncer-525n9" (UID: "79e51bb5-b3d5-4003-8c70-f789f06b54e8") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:20.310100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:20.310067 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvgk2\" (UniqueName: \"kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2\") pod \"network-check-target-cxfjn\" (UID: \"2939a38f-6e71-4bfa-86ea-82969887427d\") " pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:20.310265 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:20.310245 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:48:20.310307 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:20.310271 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:48:20.310307 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:20.310282 2548 projected.go:196] Error preparing data for projected volume kube-api-access-mvgk2 for pod openshift-network-diagnostics/network-check-target-cxfjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:20.310362 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:20.310332 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2 podName:2939a38f-6e71-4bfa-86ea-82969887427d nodeName:}" failed. No retries permitted until 2026-07-18 01:48:28.310316152 +0000 UTC m=+17.163059474 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mvgk2" (UniqueName: "kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2") pod "network-check-target-cxfjn" (UID: "2939a38f-6e71-4bfa-86ea-82969887427d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:20.543974 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:20.543256 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:20.543974 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:20.543381 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:20.543974 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:20.543731 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:20.543974 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:20.543893 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:20.543974 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:20.543949 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:20.544317 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:20.544032 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:22.543444 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:22.542840 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:22.543444 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:22.542966 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:22.543444 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:22.543361 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:22.543444 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:22.543398 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:22.544073 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:22.543533 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:22.544073 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:22.543611 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:24.542874 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:24.542804 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:24.542874 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:24.542843 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:24.543310 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:24.542913 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:24.543310 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:24.542928 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:24.543310 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:24.543018 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:24.543310 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:24.543094 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:26.543010 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:26.542976 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:26.543460 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:26.543013 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:26.543460 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:26.543018 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:26.543460 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:26.543104 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:26.543460 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:26.543193 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:26.543460 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:26.543250 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:28.173264 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:28.173233 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs\") pod \"network-metrics-daemon-7d9w7\" (UID: \"fea0f11b-353d-4fe1-86bb-08286c333fd3\") " pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:28.173759 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:28.173402 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:28.173759 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:28.173479 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs podName:fea0f11b-353d-4fe1-86bb-08286c333fd3 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:44.173456035 +0000 UTC m=+33.026199361 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs") pod "network-metrics-daemon-7d9w7" (UID: "fea0f11b-353d-4fe1-86bb-08286c333fd3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:28.274426 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:28.274397 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret\") pod \"global-pull-secret-syncer-525n9\" (UID: \"79e51bb5-b3d5-4003-8c70-f789f06b54e8\") " pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:28.274586 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:28.274544 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:28.274643 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:28.274604 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret podName:79e51bb5-b3d5-4003-8c70-f789f06b54e8 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:44.27458844 +0000 UTC m=+33.127331763 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret") pod "global-pull-secret-syncer-525n9" (UID: "79e51bb5-b3d5-4003-8c70-f789f06b54e8") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:28.375013 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:28.374982 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvgk2\" (UniqueName: \"kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2\") pod \"network-check-target-cxfjn\" (UID: \"2939a38f-6e71-4bfa-86ea-82969887427d\") " pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:28.375182 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:28.375152 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:48:28.375182 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:28.375178 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:48:28.375260 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:28.375189 2548 projected.go:196] Error preparing data for projected volume kube-api-access-mvgk2 for pod openshift-network-diagnostics/network-check-target-cxfjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:28.375260 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:28.375250 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2 podName:2939a38f-6e71-4bfa-86ea-82969887427d nodeName:}" failed. No retries permitted until 2026-07-18 01:48:44.375236306 +0000 UTC m=+33.227979648 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mvgk2" (UniqueName: "kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2") pod "network-check-target-cxfjn" (UID: "2939a38f-6e71-4bfa-86ea-82969887427d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:28.543205 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:28.543135 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:28.543408 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:28.543249 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:28.543408 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:28.543256 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:28.543408 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:28.543280 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:28.543408 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:28.543363 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:28.543617 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:28.543459 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:30.543041 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:30.543007 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:30.543525 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:30.543051 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:30.543525 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:30.543148 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:30.543525 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:30.543157 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:30.543525 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:30.543277 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:30.543525 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:30.543369 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:32.542898 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.542870 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:32.543273 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:32.542995 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:32.543273 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.543084 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:32.543273 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:32.543232 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:32.543273 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.543265 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:32.543459 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:32.543347 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:32.816553 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.816337 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cdhlp" event={"ID":"6d4e467b-f727-41ee-94b4-eb64e672a7dc","Type":"ContainerStarted","Data":"63abd2c2c2051d999ce96ed59992b92bd05e70f1669d0a4eaf41e93914006f85"} Jul 18 01:48:32.818175 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.818153 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-98wvv" event={"ID":"33094a8a-fe46-41a7-ba7a-cdf29c47e2d7","Type":"ContainerStarted","Data":"643c1ea19bf551cec57aae1c687c189e1f1e2f7b15f3c3665750a2881a49ec97"} Jul 18 01:48:32.819701 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.819677 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvmsl" event={"ID":"a8061f1d-195f-4a14-a518-5b0b0275471e","Type":"ContainerStarted","Data":"0827f21fe63643910a9041e8ebf134067de2cd26be0109a15c4658133a61035b"} Jul 18 01:48:32.821312 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.821288 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" event={"ID":"342f7f5d-cf7a-4256-9525-95f7db41efbb","Type":"ContainerStarted","Data":"fb33e5bb4279fedf5bdb2222ea18c0051a2eb4502d016af10efe50941153bab2"} Jul 18 01:48:32.823543 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.823516 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" event={"ID":"cb8b4e1e-302b-4634-a387-8569ab5be169","Type":"ContainerStarted","Data":"b475e8a77e0458f9de50e429ab45b5ad80bab9377c51cda63a47b58130cdd514"} Jul 18 01:48:32.823616 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.823583 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" event={"ID":"cb8b4e1e-302b-4634-a387-8569ab5be169","Type":"ContainerStarted","Data":"3ef8dff123c524778171fa5f86955dbd77ae1e7fcf76bbc6c531241fa5fb8842"} Jul 18 01:48:32.823676 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.823616 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" event={"ID":"cb8b4e1e-302b-4634-a387-8569ab5be169","Type":"ContainerStarted","Data":"6213ce7d382890afbc78e01462ea5230a55a76e2b59ec904304167d2071d0897"} Jul 18 01:48:32.825250 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.825228 2548 generic.go:356] "Generic (PLEG): container finished" podID="1ebbea173dadad9136c4ee6a95a9762e" containerID="e7ae0aff009c7dc03a5261b131e72f0f18a96e4fa86a8eaf8322d6395440319d" exitCode=0 Jul 18 01:48:32.825335 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.825291 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" event={"ID":"1ebbea173dadad9136c4ee6a95a9762e","Type":"ContainerDied","Data":"e7ae0aff009c7dc03a5261b131e72f0f18a96e4fa86a8eaf8322d6395440319d"} Jul 18 01:48:32.827115 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.827094 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" event={"ID":"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0","Type":"ContainerStarted","Data":"466145f049ab04b0a365653314bfbe684188d4fedd4f332f68e1d281dedf85d4"} Jul 18 01:48:32.828459 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.828438 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" event={"ID":"3d448a47aa0f5878864b37d16dc8c112","Type":"ContainerStarted","Data":"994404edc5e3a2dc5edba7ae94e9464ecb27ee5c8e25bfb8014eb26801775f06"} Jul 18 01:48:32.829271 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.829236 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cdhlp" podStartSLOduration=2.24622063 podStartE2EDuration="21.829222462s" totalImagesPullingTime="19.583001832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:32.82911067 +0000 UTC m=+21.681854014" watchObservedRunningTime="2026-07-18 01:48:32.829222462 +0000 UTC m=+21.681965807" Jul 18 01:48:32.830004 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.829948 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5n94l" event={"ID":"2ded5998-7e45-4049-9302-bab9411039ba","Type":"ContainerStarted","Data":"6b2acfe81dcb76838f5d6ac9bcf8638be5c54845a5fc2f405ce670fed09a10a7"} Jul 18 01:48:32.831155 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.831138 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bwx4x" event={"ID":"2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b","Type":"ContainerStarted","Data":"f56b2626893a8db88108f8caf6a0db2dba03b1ac6e0c055f4ecedd011451e354"} Jul 18 01:48:32.841495 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.841459 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-108.ec2.internal" podStartSLOduration=20.841447239 podStartE2EDuration="20.841447239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:32.841327083 +0000 UTC m=+21.694070427" watchObservedRunningTime="2026-07-18 01:48:32.841447239 +0000 UTC m=+21.694190583" Jul 18 01:48:32.857828 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.857770 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-4fcgn" podStartSLOduration=2.241991001 podStartE2EDuration="21.857758508s" totalImagesPullingTime="19.615767507s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:32.857610442 +0000 UTC m=+21.710353786" watchObservedRunningTime="2026-07-18 01:48:32.857758508 +0000 UTC m=+21.710501851" Jul 18 01:48:32.873257 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.873220 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-98wvv" podStartSLOduration=2.204033083 podStartE2EDuration="21.873208498s" totalImagesPullingTime="19.669175415s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:32.872924836 +0000 UTC m=+21.725668181" watchObservedRunningTime="2026-07-18 01:48:32.873208498 +0000 UTC m=+21.725951842" Jul 18 01:48:32.920928 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.920894 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bwx4x" podStartSLOduration=2.341763943 podStartE2EDuration="21.920883053s" totalImagesPullingTime="19.57911911s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:32.920591264 +0000 UTC m=+21.773334607" watchObservedRunningTime="2026-07-18 01:48:32.920883053 +0000 UTC m=+21.773626396" Jul 18 01:48:32.934086 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:32.934056 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5n94l" podStartSLOduration=2.3806806529999998 podStartE2EDuration="21.934044927s" totalImagesPullingTime="19.553364274s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:32.933640235 +0000 UTC m=+21.786383580" watchObservedRunningTime="2026-07-18 01:48:32.934044927 +0000 UTC m=+21.786788271" Jul 18 01:48:33.834915 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:33.834677 2548 generic.go:356] "Generic (PLEG): container finished" podID="a8061f1d-195f-4a14-a518-5b0b0275471e" containerID="0827f21fe63643910a9041e8ebf134067de2cd26be0109a15c4658133a61035b" exitCode=0 Jul 18 01:48:33.835650 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:33.834763 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvmsl" event={"ID":"a8061f1d-195f-4a14-a518-5b0b0275471e","Type":"ContainerDied","Data":"0827f21fe63643910a9041e8ebf134067de2cd26be0109a15c4658133a61035b"} Jul 18 01:48:33.837677 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:33.837646 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" event={"ID":"cb8b4e1e-302b-4634-a387-8569ab5be169","Type":"ContainerStarted","Data":"a2f10291cee9c8f5891e8c23bd3b9ecaa17a2a944186810bf3277c7f8670e813"} Jul 18 01:48:33.837812 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:33.837718 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" event={"ID":"cb8b4e1e-302b-4634-a387-8569ab5be169","Type":"ContainerStarted","Data":"3fcdec2a2f4d6363e65b3a3942173b9046f4c1b927202b779f2ae28be5fce137"} Jul 18 01:48:33.837812 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:33.837756 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" event={"ID":"cb8b4e1e-302b-4634-a387-8569ab5be169","Type":"ContainerStarted","Data":"e673acc6ba71dce543b71c9a1c1a66fbf949ba4151374cd818adba5695f525c3"} Jul 18 01:48:33.839080 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:33.839015 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4j47l" event={"ID":"c8f58c36-699d-4215-9b30-98e166a03995","Type":"ContainerStarted","Data":"020c29d0a2a90ef8725fa95996f8b078d9a6e4dbcae7d16aa41220ccf98f7099"} Jul 18 01:48:33.885230 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:33.885191 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4j47l" podStartSLOduration=2.3651795890000002 podStartE2EDuration="21.88517836s" totalImagesPullingTime="19.519998771s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:33.885065117 +0000 UTC m=+22.737808461" watchObservedRunningTime="2026-07-18 01:48:33.88517836 +0000 UTC m=+22.737921704" Jul 18 01:48:34.100373 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:34.100250 2548 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 18 01:48:34.516990 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:34.516881 2548 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-18T01:48:34.100270935Z","UUID":"139cbfd6-9df0-4e98-90d5-d5e802857191","Handler":null,"Name":"","Endpoint":""} Jul 18 01:48:34.518471 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:34.518448 2548 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 18 01:48:34.518603 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:34.518485 2548 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 18 01:48:34.543128 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:34.543105 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:34.543242 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:34.543198 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:34.543242 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:34.543212 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:34.543242 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:34.543221 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:34.543383 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:34.543279 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:34.543434 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:34.543379 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:34.658126 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:34.658102 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cdhlp" Jul 18 01:48:34.658901 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:34.658878 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cdhlp" Jul 18 01:48:34.843541 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:34.843469 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" event={"ID":"1ebbea173dadad9136c4ee6a95a9762e","Type":"ContainerStarted","Data":"2e0c339d91e032591048cb5f8ebd0f4d64fb45c509a445db9052f52876080fa0"} Jul 18 01:48:34.845726 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:34.845689 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" event={"ID":"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0","Type":"ContainerStarted","Data":"ffe7a5f3b1c584f00edefeba6173655f409081a2683aac3689323e2d9163b65b"} Jul 18 01:48:34.857529 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:34.857472 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-108.ec2.internal" podStartSLOduration=22.857452112 podStartE2EDuration="22.857452112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:34.857410365 +0000 UTC m=+23.710153710" watchObservedRunningTime="2026-07-18 01:48:34.857452112 +0000 UTC m=+23.710195438" Jul 18 01:48:35.850318 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:35.850236 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" event={"ID":"43fed8c5-94c5-4b8f-ac7c-91e2f4e3a1a0","Type":"ContainerStarted","Data":"6805021ea2ffc364f7565fd6b785333a1770e946d0e2f78a17db9ac6db4b3da9"} Jul 18 01:48:35.854048 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:35.854024 2548 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 18 01:48:35.854175 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:35.854022 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" event={"ID":"cb8b4e1e-302b-4634-a387-8569ab5be169","Type":"ContainerStarted","Data":"a89d6308716f0f56f758ad66a29a13d4d2dff71dfa44787d1a52fbb78a32d90c"} Jul 18 01:48:36.543089 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:36.543054 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:36.543089 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:36.543076 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:36.543384 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:36.543169 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:36.543384 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:36.543166 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:36.543384 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:36.543243 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:36.543384 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:36.543327 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:36.808847 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:36.808762 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cdhlp" Jul 18 01:48:36.809492 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:36.809471 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cdhlp" Jul 18 01:48:36.824109 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:36.824053 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jcvl9" podStartSLOduration=3.8379981279999997 podStartE2EDuration="25.824006768s" totalImagesPullingTime="21.98600864s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:35.871178731 +0000 UTC m=+24.723922076" watchObservedRunningTime="2026-07-18 01:48:36.824006768 +0000 UTC m=+25.676750115" Jul 18 01:48:38.543265 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:38.543237 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:38.543265 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:38.543262 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:38.543720 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:38.543270 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:38.543720 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:38.543355 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:38.543720 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:38.543415 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:38.543720 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:38.543489 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:38.864059 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:38.863866 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" event={"ID":"cb8b4e1e-302b-4634-a387-8569ab5be169","Type":"ContainerStarted","Data":"821606353e9385004e34dc3891964fc812aa2382ae953f7c1ed18c9412413b83"} Jul 18 01:48:38.864165 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:38.864151 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:38.878252 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:38.878232 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:38.889491 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:38.889453 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" podStartSLOduration=8.187357256 podStartE2EDuration="27.889441532s" totalImagesPullingTime="19.702084276s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:38.889176027 +0000 UTC m=+27.741919371" watchObservedRunningTime="2026-07-18 01:48:38.889441532 +0000 UTC m=+27.742184880" Jul 18 01:48:39.866742 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:39.866707 2548 generic.go:356] "Generic (PLEG): container finished" podID="a8061f1d-195f-4a14-a518-5b0b0275471e" containerID="3cd29fe1a62b3774ec592122997a86dd41357762cb08ffe835f9057674023fc1" exitCode=0 Jul 18 01:48:39.867570 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:39.866804 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvmsl" event={"ID":"a8061f1d-195f-4a14-a518-5b0b0275471e","Type":"ContainerDied","Data":"3cd29fe1a62b3774ec592122997a86dd41357762cb08ffe835f9057674023fc1"} Jul 18 01:48:39.867570 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:39.867242 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:39.867570 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:39.867293 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:39.882055 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:39.882038 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:48:40.543280 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:40.543112 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:40.543516 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:40.543112 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:40.543516 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:40.543377 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:40.543516 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:40.543160 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:40.543516 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:40.543452 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:40.543737 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:40.543555 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:40.870911 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:40.870885 2548 generic.go:356] "Generic (PLEG): container finished" podID="a8061f1d-195f-4a14-a518-5b0b0275471e" containerID="adbe5c510f0b19713dce49081d56aeb2e8927e079e540083269a5db2027b6a30" exitCode=0 Jul 18 01:48:40.871191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:40.870972 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvmsl" event={"ID":"a8061f1d-195f-4a14-a518-5b0b0275471e","Type":"ContainerDied","Data":"adbe5c510f0b19713dce49081d56aeb2e8927e079e540083269a5db2027b6a30"} Jul 18 01:48:40.981536 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:40.981513 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-525n9"] Jul 18 01:48:40.981644 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:40.981586 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:40.981688 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:40.981660 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:40.984414 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:40.984391 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cxfjn"] Jul 18 01:48:40.984517 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:40.984472 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:40.984574 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:40.984558 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:40.987001 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:40.986981 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7d9w7"] Jul 18 01:48:40.987076 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:40.987054 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:40.987169 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:40.987149 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:42.543100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:42.543073 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:42.543529 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:42.543073 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:42.543529 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:42.543192 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:42.543529 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:42.543193 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:42.543529 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:42.543293 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:42.543529 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:42.543386 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:42.877847 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:42.877816 2548 generic.go:356] "Generic (PLEG): container finished" podID="a8061f1d-195f-4a14-a518-5b0b0275471e" containerID="728307eacfdf9b17fa1cacb5f3c50a064dd83643b6a49cff3315f5f939ee39f6" exitCode=0 Jul 18 01:48:42.877969 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:42.877853 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvmsl" event={"ID":"a8061f1d-195f-4a14-a518-5b0b0275471e","Type":"ContainerDied","Data":"728307eacfdf9b17fa1cacb5f3c50a064dd83643b6a49cff3315f5f939ee39f6"} Jul 18 01:48:44.200796 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:44.200754 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs\") pod \"network-metrics-daemon-7d9w7\" (UID: \"fea0f11b-353d-4fe1-86bb-08286c333fd3\") " pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:44.201223 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:44.200893 2548 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:44.201223 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:44.200973 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs podName:fea0f11b-353d-4fe1-86bb-08286c333fd3 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:16.200950899 +0000 UTC m=+65.053694235 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs") pod "network-metrics-daemon-7d9w7" (UID: "fea0f11b-353d-4fe1-86bb-08286c333fd3") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 18 01:48:44.302092 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:44.302022 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret\") pod \"global-pull-secret-syncer-525n9\" (UID: \"79e51bb5-b3d5-4003-8c70-f789f06b54e8\") " pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:44.302237 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:44.302150 2548 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:44.302237 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:44.302212 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret podName:79e51bb5-b3d5-4003-8c70-f789f06b54e8 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:16.302192686 +0000 UTC m=+65.154936023 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret") pod "global-pull-secret-syncer-525n9" (UID: "79e51bb5-b3d5-4003-8c70-f789f06b54e8") : object "kube-system"/"original-pull-secret" not registered Jul 18 01:48:44.403068 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:44.403041 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvgk2\" (UniqueName: \"kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2\") pod \"network-check-target-cxfjn\" (UID: \"2939a38f-6e71-4bfa-86ea-82969887427d\") " pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:44.403248 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:44.403229 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 18 01:48:44.403304 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:44.403255 2548 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 18 01:48:44.403304 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:44.403269 2548 projected.go:196] Error preparing data for projected volume kube-api-access-mvgk2 for pod openshift-network-diagnostics/network-check-target-cxfjn: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:44.403379 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:44.403334 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2 podName:2939a38f-6e71-4bfa-86ea-82969887427d nodeName:}" failed. No retries permitted until 2026-07-18 01:49:16.403316751 +0000 UTC m=+65.256060091 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mvgk2" (UniqueName: "kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2") pod "network-check-target-cxfjn" (UID: "2939a38f-6e71-4bfa-86ea-82969887427d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 18 01:48:44.543180 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:44.543148 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:44.543337 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:44.543256 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:44.543337 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:44.543263 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-525n9" podUID="79e51bb5-b3d5-4003-8c70-f789f06b54e8" Jul 18 01:48:44.543453 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:44.543264 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:44.543453 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:44.543364 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7d9w7" podUID="fea0f11b-353d-4fe1-86bb-08286c333fd3" Jul 18 01:48:44.543453 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:44.543418 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cxfjn" podUID="2939a38f-6e71-4bfa-86ea-82969887427d" Jul 18 01:48:46.454574 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.454356 2548 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-108.ec2.internal" event="NodeReady" Jul 18 01:48:46.454999 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.454666 2548 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 18 01:48:46.487914 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.487893 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b997b76d5-slg46"] Jul 18 01:48:46.493428 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.493409 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.496292 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.496217 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 18 01:48:46.496292 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.496280 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 18 01:48:46.496585 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.496563 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9244r\"" Jul 18 01:48:46.497172 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.497155 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 18 01:48:46.505213 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.505191 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-r222c"] Jul 18 01:48:46.507237 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.507221 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 18 01:48:46.509042 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.509027 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zl6kk"] Jul 18 01:48:46.509166 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.509152 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r222c" Jul 18 01:48:46.511813 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.511759 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-s99dg\"" Jul 18 01:48:46.512150 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.512134 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 18 01:48:46.512862 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.512641 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 18 01:48:46.512862 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.512650 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 18 01:48:46.513011 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.512951 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zl6kk" Jul 18 01:48:46.513211 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.513192 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b997b76d5-slg46"] Jul 18 01:48:46.515349 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.515187 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 18 01:48:46.515349 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.515200 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zdfxb\"" Jul 18 01:48:46.516149 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.516110 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 18 01:48:46.518391 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.518372 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r222c"] Jul 18 01:48:46.521499 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.521309 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zl6kk"] Jul 18 01:48:46.542980 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.542955 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:48:46.543080 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.542955 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:48:46.543139 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.542955 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:48:46.545135 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.545055 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 01:48:46.545235 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.545218 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f4m6k\"" Jul 18 01:48:46.545633 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.545558 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 18 01:48:46.545633 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.545623 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gzg6q\"" Jul 18 01:48:46.545890 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.545872 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 18 01:48:46.546345 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.546328 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 01:48:46.621636 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.621607 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-bound-sa-token\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.621741 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.621681 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/468d9d35-1c8b-404e-932d-39d32f02ea1b-image-registry-private-configuration\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.621741 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.621707 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxg67\" (UniqueName: \"kubernetes.io/projected/9b463fea-e348-40d3-8024-d09e8af88bd6-kube-api-access-gxg67\") pod \"ingress-canary-r222c\" (UID: \"9b463fea-e348-40d3-8024-d09e8af88bd6\") " pod="openshift-ingress-canary/ingress-canary-r222c" Jul 18 01:48:46.621741 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.621729 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5e4da607-7ffa-41ce-a806-b53359365b9a-config-volume\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:48:46.621912 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.621754 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfrwt\" (UniqueName: \"kubernetes.io/projected/5e4da607-7ffa-41ce-a806-b53359365b9a-kube-api-access-nfrwt\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:48:46.621912 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.621853 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/468d9d35-1c8b-404e-932d-39d32f02ea1b-installation-pull-secrets\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.621912 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.621874 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b463fea-e348-40d3-8024-d09e8af88bd6-cert\") pod \"ingress-canary-r222c\" (UID: \"9b463fea-e348-40d3-8024-d09e8af88bd6\") " pod="openshift-ingress-canary/ingress-canary-r222c" Jul 18 01:48:46.621912 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.621897 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:48:46.622095 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.621924 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-certificates\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.622095 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.621953 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/468d9d35-1c8b-404e-932d-39d32f02ea1b-ca-trust-extracted\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.622095 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.621978 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnfs2\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-kube-api-access-wnfs2\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.622095 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.622013 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/468d9d35-1c8b-404e-932d-39d32f02ea1b-trusted-ca\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.622095 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.622036 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e4da607-7ffa-41ce-a806-b53359365b9a-tmp-dir\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:48:46.622095 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.622066 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.670813 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.670765 2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cdhlp_6d4e467b-f727-41ee-94b4-eb64e672a7dc/konnectivity-agent/0.log" limit=1024 Jul 18 01:48:46.722691 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.722620 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/468d9d35-1c8b-404e-932d-39d32f02ea1b-trusted-ca\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.722691 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.722659 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e4da607-7ffa-41ce-a806-b53359365b9a-tmp-dir\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:48:46.722890 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.722692 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.722890 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.722719 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-bound-sa-token\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.722890 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.722760 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/468d9d35-1c8b-404e-932d-39d32f02ea1b-image-registry-private-configuration\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.722890 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.722803 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxg67\" (UniqueName: \"kubernetes.io/projected/9b463fea-e348-40d3-8024-d09e8af88bd6-kube-api-access-gxg67\") pod \"ingress-canary-r222c\" (UID: \"9b463fea-e348-40d3-8024-d09e8af88bd6\") " pod="openshift-ingress-canary/ingress-canary-r222c" Jul 18 01:48:46.722890 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.722823 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5e4da607-7ffa-41ce-a806-b53359365b9a-config-volume\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:48:46.722890 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.722850 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfrwt\" (UniqueName: \"kubernetes.io/projected/5e4da607-7ffa-41ce-a806-b53359365b9a-kube-api-access-nfrwt\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:48:46.723207 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.722945 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/468d9d35-1c8b-404e-932d-39d32f02ea1b-installation-pull-secrets\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.723207 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.722969 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b463fea-e348-40d3-8024-d09e8af88bd6-cert\") pod \"ingress-canary-r222c\" (UID: \"9b463fea-e348-40d3-8024-d09e8af88bd6\") " pod="openshift-ingress-canary/ingress-canary-r222c" Jul 18 01:48:46.723207 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.722989 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:48:46.723207 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.723014 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-certificates\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.723207 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:46.723036 2548 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:48:46.723207 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:46.723053 2548 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b997b76d5-slg46: secret "image-registry-tls" not found Jul 18 01:48:46.723207 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:46.723105 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls podName:468d9d35-1c8b-404e-932d-39d32f02ea1b nodeName:}" failed. No retries permitted until 2026-07-18 01:48:47.223086101 +0000 UTC m=+36.075829443 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls") pod "image-registry-b997b76d5-slg46" (UID: "468d9d35-1c8b-404e-932d-39d32f02ea1b") : secret "image-registry-tls" not found Jul 18 01:48:46.723546 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.723263 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e4da607-7ffa-41ce-a806-b53359365b9a-tmp-dir\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:48:46.723546 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.723037 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/468d9d35-1c8b-404e-932d-39d32f02ea1b-ca-trust-extracted\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.723546 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.723343 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnfs2\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-kube-api-access-wnfs2\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.723546 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.723361 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/468d9d35-1c8b-404e-932d-39d32f02ea1b-ca-trust-extracted\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.723546 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.723436 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5e4da607-7ffa-41ce-a806-b53359365b9a-config-volume\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:48:46.723546 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:46.723504 2548 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:48:46.723546 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:46.723550 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls podName:5e4da607-7ffa-41ce-a806-b53359365b9a nodeName:}" failed. No retries permitted until 2026-07-18 01:48:47.223536478 +0000 UTC m=+36.076279800 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls") pod "dns-default-zl6kk" (UID: "5e4da607-7ffa-41ce-a806-b53359365b9a") : secret "dns-default-metrics-tls" not found Jul 18 01:48:46.723902 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.723695 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/468d9d35-1c8b-404e-932d-39d32f02ea1b-trusted-ca\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.723902 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.723890 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-certificates\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.728071 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.728031 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/468d9d35-1c8b-404e-932d-39d32f02ea1b-installation-pull-secrets\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.728236 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.728217 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b463fea-e348-40d3-8024-d09e8af88bd6-cert\") pod \"ingress-canary-r222c\" (UID: \"9b463fea-e348-40d3-8024-d09e8af88bd6\") " pod="openshift-ingress-canary/ingress-canary-r222c" Jul 18 01:48:46.728639 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.728593 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/468d9d35-1c8b-404e-932d-39d32f02ea1b-image-registry-private-configuration\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.732206 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.732181 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-bound-sa-token\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.732677 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.732653 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxg67\" (UniqueName: \"kubernetes.io/projected/9b463fea-e348-40d3-8024-d09e8af88bd6-kube-api-access-gxg67\") pod \"ingress-canary-r222c\" (UID: \"9b463fea-e348-40d3-8024-d09e8af88bd6\") " pod="openshift-ingress-canary/ingress-canary-r222c" Jul 18 01:48:46.732768 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.732729 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnfs2\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-kube-api-access-wnfs2\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:46.733069 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.733046 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfrwt\" (UniqueName: \"kubernetes.io/projected/5e4da607-7ffa-41ce-a806-b53359365b9a-kube-api-access-nfrwt\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:48:46.769720 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.769698 2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cdhlp_6d4e467b-f727-41ee-94b4-eb64e672a7dc/konnectivity-agent/0.log" limit=1024 Jul 18 01:48:46.820020 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.819999 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r222c" Jul 18 01:48:46.840355 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.840336 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s"] Jul 18 01:48:46.845455 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.845437 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s" Jul 18 01:48:46.848197 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.848116 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-8zxk6"] Jul 18 01:48:46.848197 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.848165 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 18 01:48:46.848197 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.848182 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-tl7x4\"" Jul 18 01:48:46.848410 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.848210 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 18 01:48:46.848410 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.848165 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 18 01:48:46.848573 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.848548 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 18 01:48:46.851316 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.851298 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:46.853692 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.853587 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 18 01:48:46.855030 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.854243 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qnh5s"] Jul 18 01:48:46.855030 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.854355 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 18 01:48:46.855030 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.854455 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 18 01:48:46.855030 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.854740 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 18 01:48:46.855030 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.854754 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-krvtv\"" Jul 18 01:48:46.858001 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.857983 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz"] Jul 18 01:48:46.858108 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.858095 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qnh5s" Jul 18 01:48:46.860607 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.860588 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 18 01:48:46.861365 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.861161 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s"] Jul 18 01:48:46.861365 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.861160 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-677qp\"" Jul 18 01:48:46.861365 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.861228 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" Jul 18 01:48:46.861365 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.861272 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 18 01:48:46.861612 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.861384 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 18 01:48:46.862882 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.862862 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-8zxk6"] Jul 18 01:48:46.865104 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.865081 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 18 01:48:46.865207 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.865133 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 18 01:48:46.865253 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.865082 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 18 01:48:46.865332 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.865311 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dxvnk\"" Jul 18 01:48:46.865418 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.865390 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 18 01:48:46.868594 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.868573 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qnh5s"] Jul 18 01:48:46.869509 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.869491 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz"] Jul 18 01:48:46.946882 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.946852 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-4qtsz"] Jul 18 01:48:46.950243 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.950223 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4qtsz" Jul 18 01:48:46.951601 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.951584 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr"] Jul 18 01:48:46.952645 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.952619 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-x54r7\"" Jul 18 01:48:46.954492 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.954467 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-84775f69dd-wq2kv"] Jul 18 01:48:46.954614 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.954598 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" Jul 18 01:48:46.957454 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.957434 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-dhpvd"] Jul 18 01:48:46.957586 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.957564 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:46.958351 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.958326 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 18 01:48:46.958458 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.958373 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 18 01:48:46.958536 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.958332 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qhmp6\"" Jul 18 01:48:46.958878 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.958846 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 18 01:48:46.961236 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.961217 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:48:46.963277 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.963253 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 18 01:48:46.963373 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.963259 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dnd64\"" Jul 18 01:48:46.963453 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.963438 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 18 01:48:46.963514 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.963462 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 18 01:48:46.963566 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.963520 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 18 01:48:46.963851 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.963836 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zjvqn\"" Jul 18 01:48:46.963995 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.963971 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 18 01:48:46.964220 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.964202 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 18 01:48:46.969812 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.968716 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 18 01:48:46.969812 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.968852 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 18 01:48:46.969944 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.969813 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 18 01:48:46.969997 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.969946 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 18 01:48:46.973589 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.973544 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-4qtsz"] Jul 18 01:48:46.975891 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.975871 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr"] Jul 18 01:48:46.977938 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.977911 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 18 01:48:46.981088 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:46.981071 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-dhpvd"] Jul 18 01:48:47.001336 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.001316 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-84775f69dd-wq2kv"] Jul 18 01:48:47.025735 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.025710 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/31ce4668-2220-411e-929d-c6991e38932a-config\") pod \"service-ca-operator-5bcbc94c78-g9f9s\" (UID: \"31ce4668-2220-411e-929d-c6991e38932a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s" Jul 18 01:48:47.025855 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.025759 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b7250c41-3b61-4533-90d6-7382a4ea66c4-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-w98tz\" (UID: \"b7250c41-3b61-4533-90d6-7382a4ea66c4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" Jul 18 01:48:47.025855 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.025811 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfkvz\" (UniqueName: \"kubernetes.io/projected/faf8aa57-0713-4c0e-8162-ed4ccb64e765-kube-api-access-wfkvz\") pod \"volume-data-source-validator-78cf4c6f6-qnh5s\" (UID: \"faf8aa57-0713-4c0e-8162-ed4ccb64e765\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qnh5s" Jul 18 01:48:47.025855 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.025839 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6rz4\" (UniqueName: \"kubernetes.io/projected/b7250c41-3b61-4533-90d6-7382a4ea66c4-kube-api-access-h6rz4\") pod \"cluster-monitoring-operator-76cf8966f8-w98tz\" (UID: \"b7250c41-3b61-4533-90d6-7382a4ea66c4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" Jul 18 01:48:47.025990 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.025862 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-tmp\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.025990 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.025947 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s7c6\" (UniqueName: \"kubernetes.io/projected/31ce4668-2220-411e-929d-c6991e38932a-kube-api-access-2s7c6\") pod \"service-ca-operator-5bcbc94c78-g9f9s\" (UID: \"31ce4668-2220-411e-929d-c6991e38932a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s" Jul 18 01:48:47.026075 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.025985 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96svz\" (UniqueName: \"kubernetes.io/projected/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-kube-api-access-96svz\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.026075 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.026063 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-snapshots\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.026171 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.026091 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-serving-cert\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.026171 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.026139 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/31ce4668-2220-411e-929d-c6991e38932a-serving-cert\") pod \"service-ca-operator-5bcbc94c78-g9f9s\" (UID: \"31ce4668-2220-411e-929d-c6991e38932a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s" Jul 18 01:48:47.026171 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.026164 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-service-ca-bundle\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.026303 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.026212 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.026303 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.026237 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-w98tz\" (UID: \"b7250c41-3b61-4533-90d6-7382a4ea66c4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" Jul 18 01:48:47.043254 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.043228 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk"] Jul 18 01:48:47.046344 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.046319 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk" Jul 18 01:48:47.050385 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.050364 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 18 01:48:47.050502 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.050416 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 18 01:48:47.050670 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.050648 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 18 01:48:47.050765 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.050683 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-ghbbx\"" Jul 18 01:48:47.050765 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.050689 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 18 01:48:47.060026 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.060008 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk"] Jul 18 01:48:47.127026 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.126996 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-snapshots\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.127159 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127030 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8nxm\" (UniqueName: \"kubernetes.io/projected/7d9e6ac4-e8b2-40ef-88de-f70735119ce5-kube-api-access-h8nxm\") pod \"console-operator-565b7b569d-dhpvd\" (UID: \"7d9e6ac4-e8b2-40ef-88de-f70735119ce5\") " pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:48:47.127159 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127048 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-serving-cert\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.127159 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127068 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:47.127159 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127094 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7d4vr\" (UID: \"08596349-c720-4300-88ab-49a75b5e1f24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" Jul 18 01:48:47.127159 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127121 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7d9e6ac4-e8b2-40ef-88de-f70735119ce5-config\") pod \"console-operator-565b7b569d-dhpvd\" (UID: \"7d9e6ac4-e8b2-40ef-88de-f70735119ce5\") " pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:48:47.127398 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127232 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/31ce4668-2220-411e-929d-c6991e38932a-serving-cert\") pod \"service-ca-operator-5bcbc94c78-g9f9s\" (UID: \"31ce4668-2220-411e-929d-c6991e38932a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s" Jul 18 01:48:47.127398 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127254 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-service-ca-bundle\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.127398 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127289 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.127398 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127315 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d9e6ac4-e8b2-40ef-88de-f70735119ce5-trusted-ca\") pod \"console-operator-565b7b569d-dhpvd\" (UID: \"7d9e6ac4-e8b2-40ef-88de-f70735119ce5\") " pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:48:47.127398 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127343 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-w98tz\" (UID: \"b7250c41-3b61-4533-90d6-7382a4ea66c4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" Jul 18 01:48:47.127398 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127379 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:47.127669 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127407 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/31ce4668-2220-411e-929d-c6991e38932a-config\") pod \"service-ca-operator-5bcbc94c78-g9f9s\" (UID: \"31ce4668-2220-411e-929d-c6991e38932a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s" Jul 18 01:48:47.127669 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127537 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-snapshots\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.127669 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.127627 2548 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:47.127836 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127690 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsr9p\" (UniqueName: \"kubernetes.io/projected/27d13595-ae03-4459-aa8c-546b1c4500b0-kube-api-access-lsr9p\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:47.127836 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.127713 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls podName:b7250c41-3b61-4533-90d6-7382a4ea66c4 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:47.627682798 +0000 UTC m=+36.480426131 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-w98tz" (UID: "b7250c41-3b61-4533-90d6-7382a4ea66c4") : secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:47.127836 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127756 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ng25z\" (UniqueName: \"kubernetes.io/projected/47b0881b-2a96-4abf-9b73-21c4a5dcaa48-kube-api-access-ng25z\") pod \"network-check-source-7bc5b6bf85-4qtsz\" (UID: \"47b0881b-2a96-4abf-9b73-21c4a5dcaa48\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4qtsz" Jul 18 01:48:47.128056 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127879 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b7250c41-3b61-4533-90d6-7382a4ea66c4-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-w98tz\" (UID: \"b7250c41-3b61-4533-90d6-7382a4ea66c4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" Jul 18 01:48:47.128056 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127928 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfkvz\" (UniqueName: \"kubernetes.io/projected/faf8aa57-0713-4c0e-8162-ed4ccb64e765-kube-api-access-wfkvz\") pod \"volume-data-source-validator-78cf4c6f6-qnh5s\" (UID: \"faf8aa57-0713-4c0e-8162-ed4ccb64e765\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qnh5s" Jul 18 01:48:47.128056 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127959 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-default-certificate\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:47.128056 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.127995 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6rz4\" (UniqueName: \"kubernetes.io/projected/b7250c41-3b61-4533-90d6-7382a4ea66c4-kube-api-access-h6rz4\") pod \"cluster-monitoring-operator-76cf8966f8-w98tz\" (UID: \"b7250c41-3b61-4533-90d6-7382a4ea66c4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" Jul 18 01:48:47.128246 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.128067 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtm8k\" (UniqueName: \"kubernetes.io/projected/08596349-c720-4300-88ab-49a75b5e1f24-kube-api-access-wtm8k\") pod \"cluster-samples-operator-79b7c55f87-7d4vr\" (UID: \"08596349-c720-4300-88ab-49a75b5e1f24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" Jul 18 01:48:47.128246 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.128094 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/31ce4668-2220-411e-929d-c6991e38932a-config\") pod \"service-ca-operator-5bcbc94c78-g9f9s\" (UID: \"31ce4668-2220-411e-929d-c6991e38932a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s" Jul 18 01:48:47.128246 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.128101 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-tmp\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.128246 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.128163 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-stats-auth\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:47.128246 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.128207 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s7c6\" (UniqueName: \"kubernetes.io/projected/31ce4668-2220-411e-929d-c6991e38932a-kube-api-access-2s7c6\") pod \"service-ca-operator-5bcbc94c78-g9f9s\" (UID: \"31ce4668-2220-411e-929d-c6991e38932a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s" Jul 18 01:48:47.128246 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.128242 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7d9e6ac4-e8b2-40ef-88de-f70735119ce5-serving-cert\") pod \"console-operator-565b7b569d-dhpvd\" (UID: \"7d9e6ac4-e8b2-40ef-88de-f70735119ce5\") " pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:48:47.128609 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.128270 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96svz\" (UniqueName: \"kubernetes.io/projected/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-kube-api-access-96svz\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.128718 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.128668 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-tmp\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.128793 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.128680 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/b7250c41-3b61-4533-90d6-7382a4ea66c4-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-w98tz\" (UID: \"b7250c41-3b61-4533-90d6-7382a4ea66c4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" Jul 18 01:48:47.129040 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.129014 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-service-ca-bundle\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.129442 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.129423 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.129849 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.129824 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/31ce4668-2220-411e-929d-c6991e38932a-serving-cert\") pod \"service-ca-operator-5bcbc94c78-g9f9s\" (UID: \"31ce4668-2220-411e-929d-c6991e38932a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s" Jul 18 01:48:47.130006 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.129989 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-serving-cert\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.139994 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.139976 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfkvz\" (UniqueName: \"kubernetes.io/projected/faf8aa57-0713-4c0e-8162-ed4ccb64e765-kube-api-access-wfkvz\") pod \"volume-data-source-validator-78cf4c6f6-qnh5s\" (UID: \"faf8aa57-0713-4c0e-8162-ed4ccb64e765\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qnh5s" Jul 18 01:48:47.145095 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.145074 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s7c6\" (UniqueName: \"kubernetes.io/projected/31ce4668-2220-411e-929d-c6991e38932a-kube-api-access-2s7c6\") pod \"service-ca-operator-5bcbc94c78-g9f9s\" (UID: \"31ce4668-2220-411e-929d-c6991e38932a\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s" Jul 18 01:48:47.145095 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.145082 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6rz4\" (UniqueName: \"kubernetes.io/projected/b7250c41-3b61-4533-90d6-7382a4ea66c4-kube-api-access-h6rz4\") pod \"cluster-monitoring-operator-76cf8966f8-w98tz\" (UID: \"b7250c41-3b61-4533-90d6-7382a4ea66c4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" Jul 18 01:48:47.145345 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.145328 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96svz\" (UniqueName: \"kubernetes.io/projected/df23c9ba-7c94-4295-81a3-7452fb9ae1a9-kube-api-access-96svz\") pod \"insights-operator-555d9d7885-8zxk6\" (UID: \"df23c9ba-7c94-4295-81a3-7452fb9ae1a9\") " pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.160408 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.160384 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s" Jul 18 01:48:47.168888 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.168865 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-8zxk6" Jul 18 01:48:47.177599 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.177522 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qnh5s" Jul 18 01:48:47.223550 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.223519 2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cdhlp_6d4e467b-f727-41ee-94b4-eb64e672a7dc/konnectivity-agent/0.log" limit=1024 Jul 18 01:48:47.229491 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.229157 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-default-certificate\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:47.229491 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.229200 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtm8k\" (UniqueName: \"kubernetes.io/projected/08596349-c720-4300-88ab-49a75b5e1f24-kube-api-access-wtm8k\") pod \"cluster-samples-operator-79b7c55f87-7d4vr\" (UID: \"08596349-c720-4300-88ab-49a75b5e1f24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" Jul 18 01:48:47.229491 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.229235 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-64rzk\" (UID: \"2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk" Jul 18 01:48:47.229491 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.229276 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-64rzk\" (UID: \"2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk" Jul 18 01:48:47.229491 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.229309 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-stats-auth\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:47.229491 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.229354 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7d9e6ac4-e8b2-40ef-88de-f70735119ce5-serving-cert\") pod \"console-operator-565b7b569d-dhpvd\" (UID: \"7d9e6ac4-e8b2-40ef-88de-f70735119ce5\") " pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:48:47.229491 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.229408 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk5mp\" (UniqueName: \"kubernetes.io/projected/2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1-kube-api-access-xk5mp\") pod \"kube-storage-version-migrator-operator-7658d9b749-64rzk\" (UID: \"2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk" Jul 18 01:48:47.229491 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.229445 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8nxm\" (UniqueName: \"kubernetes.io/projected/7d9e6ac4-e8b2-40ef-88de-f70735119ce5-kube-api-access-h8nxm\") pod \"console-operator-565b7b569d-dhpvd\" (UID: \"7d9e6ac4-e8b2-40ef-88de-f70735119ce5\") " pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:48:47.229491 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.229472 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:47.229491 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.229496 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7d4vr\" (UID: \"08596349-c720-4300-88ab-49a75b5e1f24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" Jul 18 01:48:47.235016 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.229524 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7d9e6ac4-e8b2-40ef-88de-f70735119ce5-config\") pod \"console-operator-565b7b569d-dhpvd\" (UID: \"7d9e6ac4-e8b2-40ef-88de-f70735119ce5\") " pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:48:47.235016 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.229557 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:47.235016 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.229611 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d9e6ac4-e8b2-40ef-88de-f70735119ce5-trusted-ca\") pod \"console-operator-565b7b569d-dhpvd\" (UID: \"7d9e6ac4-e8b2-40ef-88de-f70735119ce5\") " pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:48:47.235016 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.229646 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:48:47.235016 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.229674 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:47.235016 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.229702 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsr9p\" (UniqueName: \"kubernetes.io/projected/27d13595-ae03-4459-aa8c-546b1c4500b0-kube-api-access-lsr9p\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:47.235016 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.229726 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ng25z\" (UniqueName: \"kubernetes.io/projected/47b0881b-2a96-4abf-9b73-21c4a5dcaa48-kube-api-access-ng25z\") pod \"network-check-source-7bc5b6bf85-4qtsz\" (UID: \"47b0881b-2a96-4abf-9b73-21c4a5dcaa48\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4qtsz" Jul 18 01:48:47.235016 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.230696 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle podName:27d13595-ae03-4459-aa8c-546b1c4500b0 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:47.730670094 +0000 UTC m=+36.583413437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle") pod "router-default-84775f69dd-wq2kv" (UID: "27d13595-ae03-4459-aa8c-546b1c4500b0") : configmap references non-existent config key: service-ca.crt Jul 18 01:48:47.235016 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.233798 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d9e6ac4-e8b2-40ef-88de-f70735119ce5-trusted-ca\") pod \"console-operator-565b7b569d-dhpvd\" (UID: \"7d9e6ac4-e8b2-40ef-88de-f70735119ce5\") " pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:48:47.235016 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.233904 2548 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 18 01:48:47.235016 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.233956 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls podName:08596349-c720-4300-88ab-49a75b5e1f24 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:47.733939386 +0000 UTC m=+36.586682712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7d4vr" (UID: "08596349-c720-4300-88ab-49a75b5e1f24") : secret "samples-operator-tls" not found Jul 18 01:48:47.235016 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.234519 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7d9e6ac4-e8b2-40ef-88de-f70735119ce5-config\") pod \"console-operator-565b7b569d-dhpvd\" (UID: \"7d9e6ac4-e8b2-40ef-88de-f70735119ce5\") " pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:48:47.235016 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.234628 2548 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:48:47.235016 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.234644 2548 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b997b76d5-slg46: secret "image-registry-tls" not found Jul 18 01:48:47.235016 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.234684 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls podName:468d9d35-1c8b-404e-932d-39d32f02ea1b nodeName:}" failed. No retries permitted until 2026-07-18 01:48:48.234670828 +0000 UTC m=+37.087414155 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls") pod "image-registry-b997b76d5-slg46" (UID: "468d9d35-1c8b-404e-932d-39d32f02ea1b") : secret "image-registry-tls" not found Jul 18 01:48:47.235016 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.234745 2548 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 18 01:48:47.235731 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.234795 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs podName:27d13595-ae03-4459-aa8c-546b1c4500b0 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:47.734765754 +0000 UTC m=+36.587509088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs") pod "router-default-84775f69dd-wq2kv" (UID: "27d13595-ae03-4459-aa8c-546b1c4500b0") : secret "router-metrics-certs-default" not found Jul 18 01:48:47.235731 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.234851 2548 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:48:47.235731 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.234883 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls podName:5e4da607-7ffa-41ce-a806-b53359365b9a nodeName:}" failed. No retries permitted until 2026-07-18 01:48:48.234872449 +0000 UTC m=+37.087615773 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls") pod "dns-default-zl6kk" (UID: "5e4da607-7ffa-41ce-a806-b53359365b9a") : secret "dns-default-metrics-tls" not found Jul 18 01:48:47.243383 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.238553 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-default-certificate\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:47.243383 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.239760 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-stats-auth\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:47.243383 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.241137 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7d9e6ac4-e8b2-40ef-88de-f70735119ce5-serving-cert\") pod \"console-operator-565b7b569d-dhpvd\" (UID: \"7d9e6ac4-e8b2-40ef-88de-f70735119ce5\") " pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:48:47.255714 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.255305 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8nxm\" (UniqueName: \"kubernetes.io/projected/7d9e6ac4-e8b2-40ef-88de-f70735119ce5-kube-api-access-h8nxm\") pod \"console-operator-565b7b569d-dhpvd\" (UID: \"7d9e6ac4-e8b2-40ef-88de-f70735119ce5\") " pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:48:47.257802 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.257510 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsr9p\" (UniqueName: \"kubernetes.io/projected/27d13595-ae03-4459-aa8c-546b1c4500b0-kube-api-access-lsr9p\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:47.257802 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.257766 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtm8k\" (UniqueName: \"kubernetes.io/projected/08596349-c720-4300-88ab-49a75b5e1f24-kube-api-access-wtm8k\") pod \"cluster-samples-operator-79b7c55f87-7d4vr\" (UID: \"08596349-c720-4300-88ab-49a75b5e1f24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" Jul 18 01:48:47.258811 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.258759 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ng25z\" (UniqueName: \"kubernetes.io/projected/47b0881b-2a96-4abf-9b73-21c4a5dcaa48-kube-api-access-ng25z\") pod \"network-check-source-7bc5b6bf85-4qtsz\" (UID: \"47b0881b-2a96-4abf-9b73-21c4a5dcaa48\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4qtsz" Jul 18 01:48:47.262693 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.262495 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4qtsz" Jul 18 01:48:47.287340 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.287105 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r222c"] Jul 18 01:48:47.290298 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.289359 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:48:47.294940 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:47.294905 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b463fea_e348_40d3_8024_d09e8af88bd6.slice/crio-1c4c851d50e4cacc297f68072ba84a7cb52a8898d5bd9607bfff18778d2adb44 WatchSource:0}: Error finding container 1c4c851d50e4cacc297f68072ba84a7cb52a8898d5bd9607bfff18778d2adb44: Status 404 returned error can't find the container with id 1c4c851d50e4cacc297f68072ba84a7cb52a8898d5bd9607bfff18778d2adb44 Jul 18 01:48:47.332795 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.331102 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-64rzk\" (UID: \"2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk" Jul 18 01:48:47.332795 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.331148 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-64rzk\" (UID: \"2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk" Jul 18 01:48:47.332795 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.331207 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk5mp\" (UniqueName: \"kubernetes.io/projected/2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1-kube-api-access-xk5mp\") pod \"kube-storage-version-migrator-operator-7658d9b749-64rzk\" (UID: \"2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk" Jul 18 01:48:47.332795 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.332443 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-64rzk\" (UID: \"2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk" Jul 18 01:48:47.336476 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.336415 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-64rzk\" (UID: \"2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk" Jul 18 01:48:47.340622 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.340593 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk5mp\" (UniqueName: \"kubernetes.io/projected/2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1-kube-api-access-xk5mp\") pod \"kube-storage-version-migrator-operator-7658d9b749-64rzk\" (UID: \"2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk" Jul 18 01:48:47.348265 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.348238 2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cdhlp_6d4e467b-f727-41ee-94b4-eb64e672a7dc/konnectivity-agent/0.log" limit=1024 Jul 18 01:48:47.358070 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.356061 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk" Jul 18 01:48:47.362329 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.359864 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s"] Jul 18 01:48:47.364159 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:47.364105 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31ce4668_2220_411e_929d_c6991e38932a.slice/crio-6d8ed32f809be3e57b3cb86ab5f3ebf582f2dc6ff03cf60b94adf73d2135be13 WatchSource:0}: Error finding container 6d8ed32f809be3e57b3cb86ab5f3ebf582f2dc6ff03cf60b94adf73d2135be13: Status 404 returned error can't find the container with id 6d8ed32f809be3e57b3cb86ab5f3ebf582f2dc6ff03cf60b94adf73d2135be13 Jul 18 01:48:47.367410 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.367336 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-8zxk6"] Jul 18 01:48:47.371258 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:47.371230 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf23c9ba_7c94_4295_81a3_7452fb9ae1a9.slice/crio-6d235a9fcfbda467a56204a3cd6b13626d84a367a54695052637757f8f75bfef WatchSource:0}: Error finding container 6d235a9fcfbda467a56204a3cd6b13626d84a367a54695052637757f8f75bfef: Status 404 returned error can't find the container with id 6d235a9fcfbda467a56204a3cd6b13626d84a367a54695052637757f8f75bfef Jul 18 01:48:47.391570 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.391454 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qnh5s"] Jul 18 01:48:47.447657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.447633 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-4qtsz"] Jul 18 01:48:47.476263 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.476237 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-dhpvd"] Jul 18 01:48:47.479444 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:47.479357 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47b0881b_2a96_4abf_9b73_21c4a5dcaa48.slice/crio-a3e48099343755ce94433388a56ce52294d727f4ef46ebfcedd26072079bb14c WatchSource:0}: Error finding container a3e48099343755ce94433388a56ce52294d727f4ef46ebfcedd26072079bb14c: Status 404 returned error can't find the container with id a3e48099343755ce94433388a56ce52294d727f4ef46ebfcedd26072079bb14c Jul 18 01:48:47.480701 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:47.480663 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaf8aa57_0713_4c0e_8162_ed4ccb64e765.slice/crio-247af4d8ef73eac31b5efe2b80e49b3e326fb0069d99280060d078eb62fb750b WatchSource:0}: Error finding container 247af4d8ef73eac31b5efe2b80e49b3e326fb0069d99280060d078eb62fb750b: Status 404 returned error can't find the container with id 247af4d8ef73eac31b5efe2b80e49b3e326fb0069d99280060d078eb62fb750b Jul 18 01:48:47.481402 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:47.481365 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d9e6ac4_e8b2_40ef_88de_f70735119ce5.slice/crio-8e611bf126905827661646a0ec7259cd70cd4f0ecf39831f8b6f689f1ab78941 WatchSource:0}: Error finding container 8e611bf126905827661646a0ec7259cd70cd4f0ecf39831f8b6f689f1ab78941: Status 404 returned error can't find the container with id 8e611bf126905827661646a0ec7259cd70cd4f0ecf39831f8b6f689f1ab78941 Jul 18 01:48:47.593367 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.593338 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk"] Jul 18 01:48:47.596146 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:48:47.596119 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dee6aa6_deaa_4a24_bc80_93fbba3bf9d1.slice/crio-64c422eddc15c9342bcf97a1936b7c55cfc07af1a174374474d9809824872b78 WatchSource:0}: Error finding container 64c422eddc15c9342bcf97a1936b7c55cfc07af1a174374474d9809824872b78: Status 404 returned error can't find the container with id 64c422eddc15c9342bcf97a1936b7c55cfc07af1a174374474d9809824872b78 Jul 18 01:48:47.633489 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.633466 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-w98tz\" (UID: \"b7250c41-3b61-4533-90d6-7382a4ea66c4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" Jul 18 01:48:47.633612 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.633597 2548 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:47.633664 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.633655 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls podName:b7250c41-3b61-4533-90d6-7382a4ea66c4 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:48.633639976 +0000 UTC m=+37.486383297 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-w98tz" (UID: "b7250c41-3b61-4533-90d6-7382a4ea66c4") : secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:47.734741 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.734691 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:47.734741 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.734720 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7d4vr\" (UID: \"08596349-c720-4300-88ab-49a75b5e1f24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" Jul 18 01:48:47.734879 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.734839 2548 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 18 01:48:47.734879 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.734865 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle podName:27d13595-ae03-4459-aa8c-546b1c4500b0 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:48.734848338 +0000 UTC m=+37.587591663 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle") pod "router-default-84775f69dd-wq2kv" (UID: "27d13595-ae03-4459-aa8c-546b1c4500b0") : configmap references non-existent config key: service-ca.crt Jul 18 01:48:47.734965 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.734885 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls podName:08596349-c720-4300-88ab-49a75b5e1f24 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:48.734875903 +0000 UTC m=+37.587619226 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7d4vr" (UID: "08596349-c720-4300-88ab-49a75b5e1f24") : secret "samples-operator-tls" not found Jul 18 01:48:47.836216 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.836185 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:47.836388 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.836357 2548 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 18 01:48:47.836498 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:47.836431 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs podName:27d13595-ae03-4459-aa8c-546b1c4500b0 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:48.836410541 +0000 UTC m=+37.689153877 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs") pod "router-default-84775f69dd-wq2kv" (UID: "27d13595-ae03-4459-aa8c-546b1c4500b0") : secret "router-metrics-certs-default" not found Jul 18 01:48:47.891396 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.891356 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r222c" event={"ID":"9b463fea-e348-40d3-8024-d09e8af88bd6","Type":"ContainerStarted","Data":"1c4c851d50e4cacc297f68072ba84a7cb52a8898d5bd9607bfff18778d2adb44"} Jul 18 01:48:47.893186 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.893160 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk" event={"ID":"2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1","Type":"ContainerStarted","Data":"64c422eddc15c9342bcf97a1936b7c55cfc07af1a174374474d9809824872b78"} Jul 18 01:48:47.894730 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.894705 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4qtsz" event={"ID":"47b0881b-2a96-4abf-9b73-21c4a5dcaa48","Type":"ContainerStarted","Data":"a3e48099343755ce94433388a56ce52294d727f4ef46ebfcedd26072079bb14c"} Jul 18 01:48:47.896558 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.896515 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-8zxk6" event={"ID":"df23c9ba-7c94-4295-81a3-7452fb9ae1a9","Type":"ContainerStarted","Data":"6d235a9fcfbda467a56204a3cd6b13626d84a367a54695052637757f8f75bfef"} Jul 18 01:48:47.901128 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.901101 2548 generic.go:356] "Generic (PLEG): container finished" podID="a8061f1d-195f-4a14-a518-5b0b0275471e" containerID="61782b11edbdf3c2b17fbac87ddee95b26839cc8da1d7478deab6797b22f627b" exitCode=0 Jul 18 01:48:47.901266 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.901189 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvmsl" event={"ID":"a8061f1d-195f-4a14-a518-5b0b0275471e","Type":"ContainerDied","Data":"61782b11edbdf3c2b17fbac87ddee95b26839cc8da1d7478deab6797b22f627b"} Jul 18 01:48:47.903524 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.903496 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" event={"ID":"7d9e6ac4-e8b2-40ef-88de-f70735119ce5","Type":"ContainerStarted","Data":"8e611bf126905827661646a0ec7259cd70cd4f0ecf39831f8b6f689f1ab78941"} Jul 18 01:48:47.905511 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.905099 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qnh5s" event={"ID":"faf8aa57-0713-4c0e-8162-ed4ccb64e765","Type":"ContainerStarted","Data":"247af4d8ef73eac31b5efe2b80e49b3e326fb0069d99280060d078eb62fb750b"} Jul 18 01:48:47.906487 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:47.906440 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s" event={"ID":"31ce4668-2220-411e-929d-c6991e38932a","Type":"ContainerStarted","Data":"6d8ed32f809be3e57b3cb86ab5f3ebf582f2dc6ff03cf60b94adf73d2135be13"} Jul 18 01:48:48.243496 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:48.242643 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:48.243496 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:48.242723 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:48:48.243496 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:48.242939 2548 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:48:48.243496 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:48.243002 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls podName:5e4da607-7ffa-41ce-a806-b53359365b9a nodeName:}" failed. No retries permitted until 2026-07-18 01:48:50.242982675 +0000 UTC m=+39.095726001 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls") pod "dns-default-zl6kk" (UID: "5e4da607-7ffa-41ce-a806-b53359365b9a") : secret "dns-default-metrics-tls" not found Jul 18 01:48:48.243496 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:48.243079 2548 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:48:48.243496 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:48.243090 2548 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b997b76d5-slg46: secret "image-registry-tls" not found Jul 18 01:48:48.243496 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:48.243122 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls podName:468d9d35-1c8b-404e-932d-39d32f02ea1b nodeName:}" failed. No retries permitted until 2026-07-18 01:48:50.243111328 +0000 UTC m=+39.095854667 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls") pod "image-registry-b997b76d5-slg46" (UID: "468d9d35-1c8b-404e-932d-39d32f02ea1b") : secret "image-registry-tls" not found Jul 18 01:48:48.647169 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:48.646351 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-w98tz\" (UID: \"b7250c41-3b61-4533-90d6-7382a4ea66c4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" Jul 18 01:48:48.647169 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:48.646669 2548 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:48.647169 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:48.646739 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls podName:b7250c41-3b61-4533-90d6-7382a4ea66c4 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:50.646718062 +0000 UTC m=+39.499461407 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-w98tz" (UID: "b7250c41-3b61-4533-90d6-7382a4ea66c4") : secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:48.747423 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:48.747382 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:48.747602 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:48.747439 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7d4vr\" (UID: \"08596349-c720-4300-88ab-49a75b5e1f24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" Jul 18 01:48:48.747709 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:48.747675 2548 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 18 01:48:48.747795 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:48.747758 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls podName:08596349-c720-4300-88ab-49a75b5e1f24 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:50.747738564 +0000 UTC m=+39.600481886 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7d4vr" (UID: "08596349-c720-4300-88ab-49a75b5e1f24") : secret "samples-operator-tls" not found Jul 18 01:48:48.748217 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:48.748199 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle podName:27d13595-ae03-4459-aa8c-546b1c4500b0 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:50.748183177 +0000 UTC m=+39.600926504 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle") pod "router-default-84775f69dd-wq2kv" (UID: "27d13595-ae03-4459-aa8c-546b1c4500b0") : configmap references non-existent config key: service-ca.crt Jul 18 01:48:48.849663 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:48.848985 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:48.849663 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:48.849268 2548 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 18 01:48:48.849663 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:48.849338 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs podName:27d13595-ae03-4459-aa8c-546b1c4500b0 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:50.849317965 +0000 UTC m=+39.702061292 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs") pod "router-default-84775f69dd-wq2kv" (UID: "27d13595-ae03-4459-aa8c-546b1c4500b0") : secret "router-metrics-certs-default" not found Jul 18 01:48:48.924150 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:48.923297 2548 generic.go:356] "Generic (PLEG): container finished" podID="a8061f1d-195f-4a14-a518-5b0b0275471e" containerID="45444bbb0e15f797eb55f73c93ddfefbee6c72aacc215b26849bb9e35eb59670" exitCode=0 Jul 18 01:48:48.924150 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:48.923374 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvmsl" event={"ID":"a8061f1d-195f-4a14-a518-5b0b0275471e","Type":"ContainerDied","Data":"45444bbb0e15f797eb55f73c93ddfefbee6c72aacc215b26849bb9e35eb59670"} Jul 18 01:48:50.263768 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:50.263731 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:50.264146 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:50.263885 2548 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:48:50.264146 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:50.263907 2548 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b997b76d5-slg46: secret "image-registry-tls" not found Jul 18 01:48:50.264146 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:50.263947 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:48:50.264146 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:50.263965 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls podName:468d9d35-1c8b-404e-932d-39d32f02ea1b nodeName:}" failed. No retries permitted until 2026-07-18 01:48:54.263943703 +0000 UTC m=+43.116687027 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls") pod "image-registry-b997b76d5-slg46" (UID: "468d9d35-1c8b-404e-932d-39d32f02ea1b") : secret "image-registry-tls" not found Jul 18 01:48:50.264146 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:50.264032 2548 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:48:50.264146 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:50.264077 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls podName:5e4da607-7ffa-41ce-a806-b53359365b9a nodeName:}" failed. No retries permitted until 2026-07-18 01:48:54.264062243 +0000 UTC m=+43.116805578 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls") pod "dns-default-zl6kk" (UID: "5e4da607-7ffa-41ce-a806-b53359365b9a") : secret "dns-default-metrics-tls" not found Jul 18 01:48:50.323106 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:50.323079 2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cdhlp_6d4e467b-f727-41ee-94b4-eb64e672a7dc/konnectivity-agent/0.log" limit=1024 Jul 18 01:48:50.424457 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:50.424427 2548 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cdhlp_6d4e467b-f727-41ee-94b4-eb64e672a7dc/konnectivity-agent/0.log" limit=1024 Jul 18 01:48:50.667581 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:50.667406 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-w98tz\" (UID: \"b7250c41-3b61-4533-90d6-7382a4ea66c4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" Jul 18 01:48:50.667760 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:50.667585 2548 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:50.667760 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:50.667682 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls podName:b7250c41-3b61-4533-90d6-7382a4ea66c4 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:54.667659213 +0000 UTC m=+43.520402553 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-w98tz" (UID: "b7250c41-3b61-4533-90d6-7382a4ea66c4") : secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:50.769048 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:50.769011 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:50.769048 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:50.769059 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7d4vr\" (UID: \"08596349-c720-4300-88ab-49a75b5e1f24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" Jul 18 01:48:50.769286 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:50.769196 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle podName:27d13595-ae03-4459-aa8c-546b1c4500b0 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:54.769172999 +0000 UTC m=+43.621916326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle") pod "router-default-84775f69dd-wq2kv" (UID: "27d13595-ae03-4459-aa8c-546b1c4500b0") : configmap references non-existent config key: service-ca.crt Jul 18 01:48:50.769286 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:50.769200 2548 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 18 01:48:50.769286 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:50.769247 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls podName:08596349-c720-4300-88ab-49a75b5e1f24 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:54.769237482 +0000 UTC m=+43.621980818 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7d4vr" (UID: "08596349-c720-4300-88ab-49a75b5e1f24") : secret "samples-operator-tls" not found Jul 18 01:48:50.870572 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:50.870536 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:50.870746 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:50.870713 2548 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 18 01:48:50.870836 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:50.870807 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs podName:27d13595-ae03-4459-aa8c-546b1c4500b0 nodeName:}" failed. No retries permitted until 2026-07-18 01:48:54.87077004 +0000 UTC m=+43.723513362 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs") pod "router-default-84775f69dd-wq2kv" (UID: "27d13595-ae03-4459-aa8c-546b1c4500b0") : secret "router-metrics-certs-default" not found Jul 18 01:48:54.297859 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:54.297818 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:48:54.298326 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:54.297908 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:48:54.298326 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:54.297949 2548 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:48:54.298326 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:54.297967 2548 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b997b76d5-slg46: secret "image-registry-tls" not found Jul 18 01:48:54.298326 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:54.298021 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls podName:468d9d35-1c8b-404e-932d-39d32f02ea1b nodeName:}" failed. No retries permitted until 2026-07-18 01:49:02.298006237 +0000 UTC m=+51.150749559 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls") pod "image-registry-b997b76d5-slg46" (UID: "468d9d35-1c8b-404e-932d-39d32f02ea1b") : secret "image-registry-tls" not found Jul 18 01:48:54.298326 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:54.298081 2548 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:48:54.298326 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:54.298155 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls podName:5e4da607-7ffa-41ce-a806-b53359365b9a nodeName:}" failed. No retries permitted until 2026-07-18 01:49:02.298135311 +0000 UTC m=+51.150878636 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls") pod "dns-default-zl6kk" (UID: "5e4da607-7ffa-41ce-a806-b53359365b9a") : secret "dns-default-metrics-tls" not found Jul 18 01:48:54.701100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:54.701062 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-w98tz\" (UID: \"b7250c41-3b61-4533-90d6-7382a4ea66c4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" Jul 18 01:48:54.701296 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:54.701235 2548 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:54.701362 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:54.701323 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls podName:b7250c41-3b61-4533-90d6-7382a4ea66c4 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:02.701301755 +0000 UTC m=+51.554045080 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-w98tz" (UID: "b7250c41-3b61-4533-90d6-7382a4ea66c4") : secret "cluster-monitoring-operator-tls" not found Jul 18 01:48:54.802211 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:54.802178 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:54.802211 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:54.802215 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7d4vr\" (UID: \"08596349-c720-4300-88ab-49a75b5e1f24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" Jul 18 01:48:54.802389 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:54.802347 2548 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 18 01:48:54.802389 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:54.802348 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle podName:27d13595-ae03-4459-aa8c-546b1c4500b0 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:02.802330602 +0000 UTC m=+51.655073935 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle") pod "router-default-84775f69dd-wq2kv" (UID: "27d13595-ae03-4459-aa8c-546b1c4500b0") : configmap references non-existent config key: service-ca.crt Jul 18 01:48:54.802536 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:54.802518 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls podName:08596349-c720-4300-88ab-49a75b5e1f24 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:02.80249264 +0000 UTC m=+51.655235967 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7d4vr" (UID: "08596349-c720-4300-88ab-49a75b5e1f24") : secret "samples-operator-tls" not found Jul 18 01:48:54.903607 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:54.903589 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:48:54.903736 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:54.903720 2548 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 18 01:48:54.903815 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:54.903805 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs podName:27d13595-ae03-4459-aa8c-546b1c4500b0 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:02.903769802 +0000 UTC m=+51.756513124 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs") pod "router-default-84775f69dd-wq2kv" (UID: "27d13595-ae03-4459-aa8c-546b1c4500b0") : secret "router-metrics-certs-default" not found Jul 18 01:48:55.945629 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:55.945593 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvmsl" event={"ID":"a8061f1d-195f-4a14-a518-5b0b0275471e","Type":"ContainerStarted","Data":"1c09b83f158fa2ff1e9abb4b5532e43d4f94e3139021dfde991f66c6362ff7a5"} Jul 18 01:48:55.947416 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:55.947390 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/0.log" Jul 18 01:48:55.947555 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:55.947433 2548 generic.go:356] "Generic (PLEG): container finished" podID="7d9e6ac4-e8b2-40ef-88de-f70735119ce5" containerID="abb091c772d9a9e96ed7543646c8db93e0208e0ea1bfe0c6fdac4e82293799b5" exitCode=255 Jul 18 01:48:55.947555 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:55.947517 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" event={"ID":"7d9e6ac4-e8b2-40ef-88de-f70735119ce5","Type":"ContainerDied","Data":"abb091c772d9a9e96ed7543646c8db93e0208e0ea1bfe0c6fdac4e82293799b5"} Jul 18 01:48:55.947702 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:55.947681 2548 scope.go:117] "RemoveContainer" containerID="abb091c772d9a9e96ed7543646c8db93e0208e0ea1bfe0c6fdac4e82293799b5" Jul 18 01:48:55.949371 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:55.949349 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qnh5s" event={"ID":"faf8aa57-0713-4c0e-8162-ed4ccb64e765","Type":"ContainerStarted","Data":"62ebe32aec7d710fdbe91792938691e903e91b24202e1f040d47e16d55bf7598"} Jul 18 01:48:55.951356 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:55.951318 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s" event={"ID":"31ce4668-2220-411e-929d-c6991e38932a","Type":"ContainerStarted","Data":"c7bbcb71d2d0bf545af06da08df0fe93a5eff8dba0d3143abc1f6d3dd2b29a69"} Jul 18 01:48:55.952964 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:55.952937 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r222c" event={"ID":"9b463fea-e348-40d3-8024-d09e8af88bd6","Type":"ContainerStarted","Data":"67ec0b848cb2bbee16cf7085a33cce28739b9969c5a7f485c3a47b41bac69a87"} Jul 18 01:48:55.955028 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:55.955005 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk" event={"ID":"2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1","Type":"ContainerStarted","Data":"66957fc81a1df43964ae1125a854a4c8bd253c795b59b7f1428468e3da3aea05"} Jul 18 01:48:55.956501 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:55.956479 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4qtsz" event={"ID":"47b0881b-2a96-4abf-9b73-21c4a5dcaa48","Type":"ContainerStarted","Data":"1bbb25b939de961df236b6f6fdb87caeda33befb6af5de294884a11beb73c334"} Jul 18 01:48:55.958040 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:55.958022 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-8zxk6" event={"ID":"df23c9ba-7c94-4295-81a3-7452fb9ae1a9","Type":"ContainerStarted","Data":"8d64b1f06c05ae1662247260ad3c6e29cd06d793f11b2d7d71bf435f3b5a38ef"} Jul 18 01:48:55.971930 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:55.971660 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hvmsl" podStartSLOduration=14.135023576 podStartE2EDuration="44.971646895s" totalImagesPullingTime="30.836623319s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:55.970357929 +0000 UTC m=+44.823101274" watchObservedRunningTime="2026-07-18 01:48:55.971646895 +0000 UTC m=+44.824390240" Jul 18 01:48:55.989591 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:55.989552 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-8zxk6" podStartSLOduration=2.402912308 podStartE2EDuration="9.989538852s" totalImagesPullingTime="7.586626544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:55.988626361 +0000 UTC m=+44.841369705" watchObservedRunningTime="2026-07-18 01:48:55.989538852 +0000 UTC m=+44.842282196" Jul 18 01:48:56.029968 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.029921 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk" podStartSLOduration=1.66257283 podStartE2EDuration="9.02990181s" totalImagesPullingTime="7.36732898s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:56.012190337 +0000 UTC m=+44.864933682" watchObservedRunningTime="2026-07-18 01:48:56.02990181 +0000 UTC m=+44.882645155" Jul 18 01:48:56.048825 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.048770 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s" podStartSLOduration=2.456860551 podStartE2EDuration="10.048754974s" totalImagesPullingTime="7.591894423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:56.030008106 +0000 UTC m=+44.882751448" watchObservedRunningTime="2026-07-18 01:48:56.048754974 +0000 UTC m=+44.901498317" Jul 18 01:48:56.049054 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.049028 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-qnh5s" podStartSLOduration=2.578652943 podStartE2EDuration="10.04901707s" totalImagesPullingTime="7.470364127s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:56.048275896 +0000 UTC m=+44.901019241" watchObservedRunningTime="2026-07-18 01:48:56.04901707 +0000 UTC m=+44.901760416" Jul 18 01:48:56.066804 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.066619 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4qtsz" podStartSLOduration=2.288639256 podStartE2EDuration="10.06661096s" totalImagesPullingTime="7.777971704s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:56.066084209 +0000 UTC m=+44.918827553" watchObservedRunningTime="2026-07-18 01:48:56.06661096 +0000 UTC m=+44.919354304" Jul 18 01:48:56.156182 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.155968 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-r222c" podStartSLOduration=2.498321683 podStartE2EDuration="10.155946629s" totalImagesPullingTime="7.657624946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:48:56.126240531 +0000 UTC m=+44.978983878" watchObservedRunningTime="2026-07-18 01:48:56.155946629 +0000 UTC m=+45.008689974" Jul 18 01:48:56.648321 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.648282 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xmx42"] Jul 18 01:48:56.681306 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.681270 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xmx42"] Jul 18 01:48:56.681474 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.681408 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:56.683815 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.683791 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 18 01:48:56.683929 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.683771 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 18 01:48:56.683929 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.683882 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6l5cz\"" Jul 18 01:48:56.723737 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.723708 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/130558d4-120e-43cb-a644-e55e3321629b-crio-socket\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:56.723875 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.723744 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/130558d4-120e-43cb-a644-e55e3321629b-data-volume\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:56.723875 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.723771 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtdhx\" (UniqueName: \"kubernetes.io/projected/130558d4-120e-43cb-a644-e55e3321629b-kube-api-access-wtdhx\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:56.723996 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.723975 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/130558d4-120e-43cb-a644-e55e3321629b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:56.724081 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.724066 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:56.825151 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.825124 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/130558d4-120e-43cb-a644-e55e3321629b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:56.825259 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.825183 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:56.825259 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.825217 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/130558d4-120e-43cb-a644-e55e3321629b-crio-socket\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:56.825259 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.825232 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/130558d4-120e-43cb-a644-e55e3321629b-data-volume\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:56.825259 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:56.825232 2548 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 01:48:56.825259 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.825246 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtdhx\" (UniqueName: \"kubernetes.io/projected/130558d4-120e-43cb-a644-e55e3321629b-kube-api-access-wtdhx\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:56.825525 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:56.825282 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/130558d4-120e-43cb-a644-e55e3321629b-kube-rbac-proxy-cm podName:130558d4-120e-43cb-a644-e55e3321629b nodeName:}" failed. No retries permitted until 2026-07-18 01:48:57.325267119 +0000 UTC m=+46.178010441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/130558d4-120e-43cb-a644-e55e3321629b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xmx42" (UID: "130558d4-120e-43cb-a644-e55e3321629b") : configmap "kube-rbac-proxy" not found Jul 18 01:48:56.825525 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.825321 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/130558d4-120e-43cb-a644-e55e3321629b-crio-socket\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:56.825525 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:56.825358 2548 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:48:56.825525 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:56.825413 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls podName:130558d4-120e-43cb-a644-e55e3321629b nodeName:}" failed. No retries permitted until 2026-07-18 01:48:57.325395522 +0000 UTC m=+46.178138845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xmx42" (UID: "130558d4-120e-43cb-a644-e55e3321629b") : secret "insights-runtime-extractor-tls" not found Jul 18 01:48:56.825736 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.825536 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/130558d4-120e-43cb-a644-e55e3321629b-data-volume\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:56.834221 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.834200 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtdhx\" (UniqueName: \"kubernetes.io/projected/130558d4-120e-43cb-a644-e55e3321629b-kube-api-access-wtdhx\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:56.962653 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.962599 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/1.log" Jul 18 01:48:56.963034 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.962986 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/0.log" Jul 18 01:48:56.963034 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.963015 2548 generic.go:356] "Generic (PLEG): container finished" podID="7d9e6ac4-e8b2-40ef-88de-f70735119ce5" containerID="86d189f977dc659cc67be613f4d815fb7482942914f9865e83105dad6e77f467" exitCode=255 Jul 18 01:48:56.963150 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.963043 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" event={"ID":"7d9e6ac4-e8b2-40ef-88de-f70735119ce5","Type":"ContainerDied","Data":"86d189f977dc659cc67be613f4d815fb7482942914f9865e83105dad6e77f467"} Jul 18 01:48:56.963150 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.963115 2548 scope.go:117] "RemoveContainer" containerID="abb091c772d9a9e96ed7543646c8db93e0208e0ea1bfe0c6fdac4e82293799b5" Jul 18 01:48:56.963365 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:56.963348 2548 scope.go:117] "RemoveContainer" containerID="86d189f977dc659cc67be613f4d815fb7482942914f9865e83105dad6e77f467" Jul 18 01:48:56.963538 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:56.963518 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-dhpvd_openshift-console-operator(7d9e6ac4-e8b2-40ef-88de-f70735119ce5)\"" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" podUID="7d9e6ac4-e8b2-40ef-88de-f70735119ce5" Jul 18 01:48:57.290025 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:57.289954 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:48:57.290025 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:57.290008 2548 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:48:57.329727 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:57.329702 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/130558d4-120e-43cb-a644-e55e3321629b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:57.329811 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:57.329763 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:57.329862 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:57.329842 2548 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 01:48:57.329914 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:57.329904 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/130558d4-120e-43cb-a644-e55e3321629b-kube-rbac-proxy-cm podName:130558d4-120e-43cb-a644-e55e3321629b nodeName:}" failed. No retries permitted until 2026-07-18 01:48:58.329889535 +0000 UTC m=+47.182632858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/130558d4-120e-43cb-a644-e55e3321629b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xmx42" (UID: "130558d4-120e-43cb-a644-e55e3321629b") : configmap "kube-rbac-proxy" not found Jul 18 01:48:57.329960 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:57.329916 2548 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:48:57.329960 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:57.329957 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls podName:130558d4-120e-43cb-a644-e55e3321629b nodeName:}" failed. No retries permitted until 2026-07-18 01:48:58.329945136 +0000 UTC m=+47.182688458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xmx42" (UID: "130558d4-120e-43cb-a644-e55e3321629b") : secret "insights-runtime-extractor-tls" not found Jul 18 01:48:57.968313 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:57.968287 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/1.log" Jul 18 01:48:57.968689 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:57.968568 2548 scope.go:117] "RemoveContainer" containerID="86d189f977dc659cc67be613f4d815fb7482942914f9865e83105dad6e77f467" Jul 18 01:48:57.968740 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:57.968694 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-dhpvd_openshift-console-operator(7d9e6ac4-e8b2-40ef-88de-f70735119ce5)\"" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" podUID="7d9e6ac4-e8b2-40ef-88de-f70735119ce5" Jul 18 01:48:58.242549 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:58.242477 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bwx4x_2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b/dns-node-resolver/0.log" Jul 18 01:48:58.339337 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:58.339310 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:58.339466 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:58.339411 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/130558d4-120e-43cb-a644-e55e3321629b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:48:58.339466 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:58.339446 2548 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:48:58.339545 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:58.339483 2548 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 01:48:58.339545 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:58.339500 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls podName:130558d4-120e-43cb-a644-e55e3321629b nodeName:}" failed. No retries permitted until 2026-07-18 01:49:00.339487397 +0000 UTC m=+49.192230719 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xmx42" (UID: "130558d4-120e-43cb-a644-e55e3321629b") : secret "insights-runtime-extractor-tls" not found Jul 18 01:48:58.339545 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:58.339520 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/130558d4-120e-43cb-a644-e55e3321629b-kube-rbac-proxy-cm podName:130558d4-120e-43cb-a644-e55e3321629b nodeName:}" failed. No retries permitted until 2026-07-18 01:49:00.339509397 +0000 UTC m=+49.192252719 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/130558d4-120e-43cb-a644-e55e3321629b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xmx42" (UID: "130558d4-120e-43cb-a644-e55e3321629b") : configmap "kube-rbac-proxy" not found Jul 18 01:48:58.975356 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:58.975329 2548 scope.go:117] "RemoveContainer" containerID="86d189f977dc659cc67be613f4d815fb7482942914f9865e83105dad6e77f467" Jul 18 01:48:58.975737 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:48:58.975476 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-dhpvd_openshift-console-operator(7d9e6ac4-e8b2-40ef-88de-f70735119ce5)\"" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" podUID="7d9e6ac4-e8b2-40ef-88de-f70735119ce5" Jul 18 01:48:59.045204 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:48:59.045179 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5n94l_2ded5998-7e45-4049-9302-bab9411039ba/node-ca/0.log" Jul 18 01:49:00.043135 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:00.043102 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r222c_9b463fea-e348-40d3-8024-d09e8af88bd6/serve-healthcheck-canary/0.log" Jul 18 01:49:00.353529 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:00.353447 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/130558d4-120e-43cb-a644-e55e3321629b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:49:00.353675 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:00.353566 2548 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 18 01:49:00.353675 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:00.353612 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:49:00.353675 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:00.353620 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/130558d4-120e-43cb-a644-e55e3321629b-kube-rbac-proxy-cm podName:130558d4-120e-43cb-a644-e55e3321629b nodeName:}" failed. No retries permitted until 2026-07-18 01:49:04.353605269 +0000 UTC m=+53.206348592 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/130558d4-120e-43cb-a644-e55e3321629b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xmx42" (UID: "130558d4-120e-43cb-a644-e55e3321629b") : configmap "kube-rbac-proxy" not found Jul 18 01:49:00.353868 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:00.353759 2548 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:49:00.353868 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:00.353852 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls podName:130558d4-120e-43cb-a644-e55e3321629b nodeName:}" failed. No retries permitted until 2026-07-18 01:49:04.353834493 +0000 UTC m=+53.206577815 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xmx42" (UID: "130558d4-120e-43cb-a644-e55e3321629b") : secret "insights-runtime-extractor-tls" not found Jul 18 01:49:02.369799 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:02.369744 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:49:02.370231 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:02.369909 2548 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 18 01:49:02.370231 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:02.369949 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:49:02.370231 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:02.369978 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls podName:5e4da607-7ffa-41ce-a806-b53359365b9a nodeName:}" failed. No retries permitted until 2026-07-18 01:49:18.369957145 +0000 UTC m=+67.222700482 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls") pod "dns-default-zl6kk" (UID: "5e4da607-7ffa-41ce-a806-b53359365b9a") : secret "dns-default-metrics-tls" not found Jul 18 01:49:02.370231 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:02.370019 2548 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 18 01:49:02.370231 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:02.370030 2548 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b997b76d5-slg46: secret "image-registry-tls" not found Jul 18 01:49:02.370231 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:02.370078 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls podName:468d9d35-1c8b-404e-932d-39d32f02ea1b nodeName:}" failed. No retries permitted until 2026-07-18 01:49:18.370068185 +0000 UTC m=+67.222811507 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls") pod "image-registry-b997b76d5-slg46" (UID: "468d9d35-1c8b-404e-932d-39d32f02ea1b") : secret "image-registry-tls" not found Jul 18 01:49:02.773072 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:02.773040 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-w98tz\" (UID: \"b7250c41-3b61-4533-90d6-7382a4ea66c4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" Jul 18 01:49:02.773245 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:02.773162 2548 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 18 01:49:02.773245 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:02.773214 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls podName:b7250c41-3b61-4533-90d6-7382a4ea66c4 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:18.773200319 +0000 UTC m=+67.625943641 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-w98tz" (UID: "b7250c41-3b61-4533-90d6-7382a4ea66c4") : secret "cluster-monitoring-operator-tls" not found Jul 18 01:49:02.874203 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:02.874173 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7d4vr\" (UID: \"08596349-c720-4300-88ab-49a75b5e1f24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" Jul 18 01:49:02.874357 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:02.874322 2548 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 18 01:49:02.874422 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:02.874385 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls podName:08596349-c720-4300-88ab-49a75b5e1f24 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:18.874371477 +0000 UTC m=+67.727114798 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7d4vr" (UID: "08596349-c720-4300-88ab-49a75b5e1f24") : secret "samples-operator-tls" not found Jul 18 01:49:02.874422 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:02.874325 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:49:02.874422 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:02.874415 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle podName:27d13595-ae03-4459-aa8c-546b1c4500b0 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:18.874390677 +0000 UTC m=+67.727134003 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle") pod "router-default-84775f69dd-wq2kv" (UID: "27d13595-ae03-4459-aa8c-546b1c4500b0") : configmap references non-existent config key: service-ca.crt Jul 18 01:49:02.975468 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:02.975429 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:49:02.975618 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:02.975553 2548 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 18 01:49:02.975618 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:02.975611 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs podName:27d13595-ae03-4459-aa8c-546b1c4500b0 nodeName:}" failed. No retries permitted until 2026-07-18 01:49:18.975596424 +0000 UTC m=+67.828339756 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs") pod "router-default-84775f69dd-wq2kv" (UID: "27d13595-ae03-4459-aa8c-546b1c4500b0") : secret "router-metrics-certs-default" not found Jul 18 01:49:04.389512 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:04.389478 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/130558d4-120e-43cb-a644-e55e3321629b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:49:04.389971 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:04.389555 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:49:04.389971 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:04.389735 2548 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 18 01:49:04.389971 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:04.389841 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls podName:130558d4-120e-43cb-a644-e55e3321629b nodeName:}" failed. No retries permitted until 2026-07-18 01:49:12.38981877 +0000 UTC m=+61.242562092 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xmx42" (UID: "130558d4-120e-43cb-a644-e55e3321629b") : secret "insights-runtime-extractor-tls" not found Jul 18 01:49:04.390130 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:04.390028 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/130558d4-120e-43cb-a644-e55e3321629b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:49:11.884154 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:11.884128 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-kq5w4" Jul 18 01:49:12.456719 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:12.456679 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:49:12.459056 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:12.459026 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/130558d4-120e-43cb-a644-e55e3321629b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xmx42\" (UID: \"130558d4-120e-43cb-a644-e55e3321629b\") " pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:49:12.594441 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:12.594413 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6l5cz\"" Jul 18 01:49:12.602291 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:12.602276 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xmx42" Jul 18 01:49:12.726287 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:12.726096 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xmx42"] Jul 18 01:49:12.728877 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:49:12.728845 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod130558d4_120e_43cb_a644_e55e3321629b.slice/crio-dd7975fcbd7a8c68480ba08f847f1ee4c057a23dbc94dbccc239914e2ab77a0a WatchSource:0}: Error finding container dd7975fcbd7a8c68480ba08f847f1ee4c057a23dbc94dbccc239914e2ab77a0a: Status 404 returned error can't find the container with id dd7975fcbd7a8c68480ba08f847f1ee4c057a23dbc94dbccc239914e2ab77a0a Jul 18 01:49:13.017290 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:13.017223 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xmx42" event={"ID":"130558d4-120e-43cb-a644-e55e3321629b","Type":"ContainerStarted","Data":"dd7975fcbd7a8c68480ba08f847f1ee4c057a23dbc94dbccc239914e2ab77a0a"} Jul 18 01:49:14.543604 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:14.543576 2548 scope.go:117] "RemoveContainer" containerID="86d189f977dc659cc67be613f4d815fb7482942914f9865e83105dad6e77f467" Jul 18 01:49:15.027724 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:15.027699 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 01:49:15.028205 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:15.028183 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/1.log" Jul 18 01:49:15.028338 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:15.028220 2548 generic.go:356] "Generic (PLEG): container finished" podID="7d9e6ac4-e8b2-40ef-88de-f70735119ce5" containerID="e69de986fb4725327a508cb371072f636047dc2df03776de7f7bd327828e7ad5" exitCode=255 Jul 18 01:49:15.028338 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:15.028281 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" event={"ID":"7d9e6ac4-e8b2-40ef-88de-f70735119ce5","Type":"ContainerDied","Data":"e69de986fb4725327a508cb371072f636047dc2df03776de7f7bd327828e7ad5"} Jul 18 01:49:15.028448 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:15.028337 2548 scope.go:117] "RemoveContainer" containerID="86d189f977dc659cc67be613f4d815fb7482942914f9865e83105dad6e77f467" Jul 18 01:49:15.028693 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:15.028667 2548 scope.go:117] "RemoveContainer" containerID="e69de986fb4725327a508cb371072f636047dc2df03776de7f7bd327828e7ad5" Jul 18 01:49:15.028909 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:15.028889 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-dhpvd_openshift-console-operator(7d9e6ac4-e8b2-40ef-88de-f70735119ce5)\"" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" podUID="7d9e6ac4-e8b2-40ef-88de-f70735119ce5" Jul 18 01:49:16.295705 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:16.295675 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs\") pod \"network-metrics-daemon-7d9w7\" (UID: \"fea0f11b-353d-4fe1-86bb-08286c333fd3\") " pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:49:16.298573 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:16.298549 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 18 01:49:16.309264 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:16.309238 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fea0f11b-353d-4fe1-86bb-08286c333fd3-metrics-certs\") pod \"network-metrics-daemon-7d9w7\" (UID: \"fea0f11b-353d-4fe1-86bb-08286c333fd3\") " pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:49:16.397137 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:16.397105 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret\") pod \"global-pull-secret-syncer-525n9\" (UID: \"79e51bb5-b3d5-4003-8c70-f789f06b54e8\") " pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:49:16.400125 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:16.400100 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 18 01:49:16.410407 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:16.410378 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/79e51bb5-b3d5-4003-8c70-f789f06b54e8-original-pull-secret\") pod \"global-pull-secret-syncer-525n9\" (UID: \"79e51bb5-b3d5-4003-8c70-f789f06b54e8\") " pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:49:16.498132 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:16.498106 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvgk2\" (UniqueName: \"kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2\") pod \"network-check-target-cxfjn\" (UID: \"2939a38f-6e71-4bfa-86ea-82969887427d\") " pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:49:16.500835 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:16.500811 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvgk2\" (UniqueName: \"kubernetes.io/projected/2939a38f-6e71-4bfa-86ea-82969887427d-kube-api-access-mvgk2\") pod \"network-check-target-cxfjn\" (UID: \"2939a38f-6e71-4bfa-86ea-82969887427d\") " pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:49:16.557457 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:16.557394 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-f4m6k\"" Jul 18 01:49:16.564344 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:16.564315 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-gzg6q\"" Jul 18 01:49:16.565302 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:16.565283 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7d9w7" Jul 18 01:49:16.568161 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:16.568141 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-525n9" Jul 18 01:49:16.573340 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:16.573318 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:49:17.036475 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:17.036450 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 01:49:17.290411 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:17.290323 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:49:17.290411 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:17.290376 2548 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:49:17.290789 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:17.290758 2548 scope.go:117] "RemoveContainer" containerID="e69de986fb4725327a508cb371072f636047dc2df03776de7f7bd327828e7ad5" Jul 18 01:49:17.291009 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:17.290982 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-dhpvd_openshift-console-operator(7d9e6ac4-e8b2-40ef-88de-f70735119ce5)\"" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" podUID="7d9e6ac4-e8b2-40ef-88de-f70735119ce5" Jul 18 01:49:17.571893 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:17.571864 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cxfjn"] Jul 18 01:49:17.575743 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:49:17.575705 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2939a38f_6e71_4bfa_86ea_82969887427d.slice/crio-fd3e3290461a3bb5918d577411dfd3e278913ebe169644e2b2a0487f37ca678f WatchSource:0}: Error finding container fd3e3290461a3bb5918d577411dfd3e278913ebe169644e2b2a0487f37ca678f: Status 404 returned error can't find the container with id fd3e3290461a3bb5918d577411dfd3e278913ebe169644e2b2a0487f37ca678f Jul 18 01:49:17.588461 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:17.588437 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7d9w7"] Jul 18 01:49:17.591597 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:49:17.591576 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfea0f11b_353d_4fe1_86bb_08286c333fd3.slice/crio-1347b49b6685a582119aa22fa96f1bc8ccbb55ba448f979f6b0740d39288e58c WatchSource:0}: Error finding container 1347b49b6685a582119aa22fa96f1bc8ccbb55ba448f979f6b0740d39288e58c: Status 404 returned error can't find the container with id 1347b49b6685a582119aa22fa96f1bc8ccbb55ba448f979f6b0740d39288e58c Jul 18 01:49:17.607359 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:17.607230 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-525n9"] Jul 18 01:49:17.609806 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:49:17.609758 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79e51bb5_b3d5_4003_8c70_f789f06b54e8.slice/crio-aa1a3309338db04cdce1b270ee1b8cd4f92c6fb667b7388ae70a16c3ca6521a6 WatchSource:0}: Error finding container aa1a3309338db04cdce1b270ee1b8cd4f92c6fb667b7388ae70a16c3ca6521a6: Status 404 returned error can't find the container with id aa1a3309338db04cdce1b270ee1b8cd4f92c6fb667b7388ae70a16c3ca6521a6 Jul 18 01:49:18.041331 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.041298 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-525n9" event={"ID":"79e51bb5-b3d5-4003-8c70-f789f06b54e8","Type":"ContainerStarted","Data":"aa1a3309338db04cdce1b270ee1b8cd4f92c6fb667b7388ae70a16c3ca6521a6"} Jul 18 01:49:18.043150 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.043118 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xmx42" event={"ID":"130558d4-120e-43cb-a644-e55e3321629b","Type":"ContainerStarted","Data":"dcf30aa742f0564979cab0ca51eae7fbacfa72f83450b399c545397fb19453f5"} Jul 18 01:49:18.044429 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.044403 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7d9w7" event={"ID":"fea0f11b-353d-4fe1-86bb-08286c333fd3","Type":"ContainerStarted","Data":"1347b49b6685a582119aa22fa96f1bc8ccbb55ba448f979f6b0740d39288e58c"} Jul 18 01:49:18.046459 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.046427 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cxfjn" event={"ID":"2939a38f-6e71-4bfa-86ea-82969887427d","Type":"ContainerStarted","Data":"835fd78f62029301b08f8156cb07d564f433c5b94871e5656c71aa4ad8a7583f"} Jul 18 01:49:18.046563 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.046468 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cxfjn" event={"ID":"2939a38f-6e71-4bfa-86ea-82969887427d","Type":"ContainerStarted","Data":"fd3e3290461a3bb5918d577411dfd3e278913ebe169644e2b2a0487f37ca678f"} Jul 18 01:49:18.047151 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.047133 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:49:18.065196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.065157 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cxfjn" podStartSLOduration=67.06514531 podStartE2EDuration="1m7.06514531s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:18.063464203 +0000 UTC m=+66.916207546" watchObservedRunningTime="2026-07-18 01:49:18.06514531 +0000 UTC m=+66.917888644" Jul 18 01:49:18.414317 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.414119 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:49:18.414317 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.414200 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:49:18.417066 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.417014 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5e4da607-7ffa-41ce-a806-b53359365b9a-metrics-tls\") pod \"dns-default-zl6kk\" (UID: \"5e4da607-7ffa-41ce-a806-b53359365b9a\") " pod="openshift-dns/dns-default-zl6kk" Jul 18 01:49:18.417652 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.417614 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls\") pod \"image-registry-b997b76d5-slg46\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:49:18.607468 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.607442 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9244r\"" Jul 18 01:49:18.615695 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.615669 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:49:18.629311 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.629290 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zdfxb\"" Jul 18 01:49:18.637386 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.637363 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zl6kk" Jul 18 01:49:18.786875 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.786851 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b997b76d5-slg46"] Jul 18 01:49:18.791940 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:49:18.790883 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod468d9d35_1c8b_404e_932d_39d32f02ea1b.slice/crio-4ba2d0a33c914468f23dc31e8f72499e413242997b33c40769c2d0c3e0bfdaca WatchSource:0}: Error finding container 4ba2d0a33c914468f23dc31e8f72499e413242997b33c40769c2d0c3e0bfdaca: Status 404 returned error can't find the container with id 4ba2d0a33c914468f23dc31e8f72499e413242997b33c40769c2d0c3e0bfdaca Jul 18 01:49:18.807205 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.807135 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zl6kk"] Jul 18 01:49:18.810184 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:49:18.810156 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e4da607_7ffa_41ce_a806_b53359365b9a.slice/crio-fe2c4a10a3cbfbfffdbd6f24aee1b0fc78c38cb6eeffbd49006f11f41b77fc18 WatchSource:0}: Error finding container fe2c4a10a3cbfbfffdbd6f24aee1b0fc78c38cb6eeffbd49006f11f41b77fc18: Status 404 returned error can't find the container with id fe2c4a10a3cbfbfffdbd6f24aee1b0fc78c38cb6eeffbd49006f11f41b77fc18 Jul 18 01:49:18.817386 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.817355 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-w98tz\" (UID: \"b7250c41-3b61-4533-90d6-7382a4ea66c4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" Jul 18 01:49:18.822178 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.822152 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/b7250c41-3b61-4533-90d6-7382a4ea66c4-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-w98tz\" (UID: \"b7250c41-3b61-4533-90d6-7382a4ea66c4\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" Jul 18 01:49:18.918520 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.918044 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:49:18.918520 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.918090 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7d4vr\" (UID: \"08596349-c720-4300-88ab-49a75b5e1f24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" Jul 18 01:49:18.918884 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.918860 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27d13595-ae03-4459-aa8c-546b1c4500b0-service-ca-bundle\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:49:18.921390 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.921313 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/08596349-c720-4300-88ab-49a75b5e1f24-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7d4vr\" (UID: \"08596349-c720-4300-88ab-49a75b5e1f24\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" Jul 18 01:49:18.986263 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.986057 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dxvnk\"" Jul 18 01:49:18.995350 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:18.994719 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" Jul 18 01:49:19.019479 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:19.019411 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:49:19.024002 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:19.023940 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/27d13595-ae03-4459-aa8c-546b1c4500b0-metrics-certs\") pod \"router-default-84775f69dd-wq2kv\" (UID: \"27d13595-ae03-4459-aa8c-546b1c4500b0\") " pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:49:19.054374 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:19.054328 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b997b76d5-slg46" event={"ID":"468d9d35-1c8b-404e-932d-39d32f02ea1b","Type":"ContainerStarted","Data":"4221d6c2e0df709b47f2e513964756ea3c918ce692864d27bb98f6ff870942fb"} Jul 18 01:49:19.054511 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:19.054406 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b997b76d5-slg46" event={"ID":"468d9d35-1c8b-404e-932d-39d32f02ea1b","Type":"ContainerStarted","Data":"4ba2d0a33c914468f23dc31e8f72499e413242997b33c40769c2d0c3e0bfdaca"} Jul 18 01:49:19.055219 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:19.055161 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:49:19.058345 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:19.058256 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7d9w7" event={"ID":"fea0f11b-353d-4fe1-86bb-08286c333fd3","Type":"ContainerStarted","Data":"1f8b52e7707f8eb77ad5f48f55907be8e762f58eeba6863e40fd99ae8eb00a87"} Jul 18 01:49:19.058345 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:19.058304 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7d9w7" event={"ID":"fea0f11b-353d-4fe1-86bb-08286c333fd3","Type":"ContainerStarted","Data":"bda63953f441936ca7b9774601f24c7c0f11b13f779c6b3e253e2003c725b0f2"} Jul 18 01:49:19.059758 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:19.059736 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zl6kk" event={"ID":"5e4da607-7ffa-41ce-a806-b53359365b9a","Type":"ContainerStarted","Data":"fe2c4a10a3cbfbfffdbd6f24aee1b0fc78c38cb6eeffbd49006f11f41b77fc18"} Jul 18 01:49:19.081674 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:19.081480 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qhmp6\"" Jul 18 01:49:19.086203 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:19.085995 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b997b76d5-slg46" podStartSLOduration=67.085976079 podStartE2EDuration="1m7.085976079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:19.084206 +0000 UTC m=+67.936949345" watchObservedRunningTime="2026-07-18 01:49:19.085976079 +0000 UTC m=+67.938719422" Jul 18 01:49:19.086203 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:19.086106 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zjvqn\"" Jul 18 01:49:19.089586 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:19.089378 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" Jul 18 01:49:19.094556 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:19.094537 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:49:19.109976 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:19.109855 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7d9w7" podStartSLOduration=66.925414671 podStartE2EDuration="1m8.109835202s" totalImagesPullingTime="1.184420531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:19.108536121 +0000 UTC m=+67.961279466" watchObservedRunningTime="2026-07-18 01:49:19.109835202 +0000 UTC m=+67.962578549" Jul 18 01:49:19.227478 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:19.227040 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz"] Jul 18 01:49:19.241944 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:49:19.241913 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7250c41_3b61_4533_90d6_7382a4ea66c4.slice/crio-741bc308a1271e876eeb9c4a8294073b0e5319730d813a5ce24db82c67f45eee WatchSource:0}: Error finding container 741bc308a1271e876eeb9c4a8294073b0e5319730d813a5ce24db82c67f45eee: Status 404 returned error can't find the container with id 741bc308a1271e876eeb9c4a8294073b0e5319730d813a5ce24db82c67f45eee Jul 18 01:49:19.386099 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:19.386068 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr"] Jul 18 01:49:19.395673 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:19.395590 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-84775f69dd-wq2kv"] Jul 18 01:49:19.409416 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:49:19.406531 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d13595_ae03_4459_aa8c_546b1c4500b0.slice/crio-f74d12d385c3756bdbd54c0c777cc7bdd5f2abe6094eceb9bd8fbcb6436121e6 WatchSource:0}: Error finding container f74d12d385c3756bdbd54c0c777cc7bdd5f2abe6094eceb9bd8fbcb6436121e6: Status 404 returned error can't find the container with id f74d12d385c3756bdbd54c0c777cc7bdd5f2abe6094eceb9bd8fbcb6436121e6 Jul 18 01:49:20.067564 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:20.067485 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" event={"ID":"b7250c41-3b61-4533-90d6-7382a4ea66c4","Type":"ContainerStarted","Data":"741bc308a1271e876eeb9c4a8294073b0e5319730d813a5ce24db82c67f45eee"} Jul 18 01:49:20.071402 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:20.070655 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-84775f69dd-wq2kv" event={"ID":"27d13595-ae03-4459-aa8c-546b1c4500b0","Type":"ContainerStarted","Data":"28d4374e81aa77505d43434d5b3d8204309de6e5a84ed78b262e55a9232b5f82"} Jul 18 01:49:20.071402 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:20.070702 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-84775f69dd-wq2kv" event={"ID":"27d13595-ae03-4459-aa8c-546b1c4500b0","Type":"ContainerStarted","Data":"f74d12d385c3756bdbd54c0c777cc7bdd5f2abe6094eceb9bd8fbcb6436121e6"} Jul 18 01:49:20.075498 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:20.074070 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" event={"ID":"08596349-c720-4300-88ab-49a75b5e1f24","Type":"ContainerStarted","Data":"fcc617df99fc0ae5a50970bf8656b427afaecb69516576aab5bd3756819278fd"} Jul 18 01:49:20.093100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:20.092827 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-84775f69dd-wq2kv" podStartSLOduration=34.092807274 podStartE2EDuration="34.092807274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:20.090918261 +0000 UTC m=+68.943661606" watchObservedRunningTime="2026-07-18 01:49:20.092807274 +0000 UTC m=+68.945550619" Jul 18 01:49:20.095232 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:20.095179 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:49:20.098451 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:20.098282 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:49:21.078053 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:21.078024 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:49:21.079468 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:21.079441 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-84775f69dd-wq2kv" Jul 18 01:49:26.100869 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:26.100686 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zl6kk" event={"ID":"5e4da607-7ffa-41ce-a806-b53359365b9a","Type":"ContainerStarted","Data":"48051b5d2247b16fa2b56aac47a5167bceadc88febc16514241925ccd769060b"} Jul 18 01:49:26.103145 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:26.102887 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" event={"ID":"b7250c41-3b61-4533-90d6-7382a4ea66c4","Type":"ContainerStarted","Data":"5de410ec9a9aff78b6ec17c2dd561b7cd3d09ea33803fff5d31a43454647e751"} Jul 18 01:49:26.105006 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:26.104939 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-525n9" event={"ID":"79e51bb5-b3d5-4003-8c70-f789f06b54e8","Type":"ContainerStarted","Data":"4cced5ed0100c61487c18d26e2ac5569855aec19143475fa2f206368515c9ddd"} Jul 18 01:49:26.107277 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:26.107219 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" event={"ID":"08596349-c720-4300-88ab-49a75b5e1f24","Type":"ContainerStarted","Data":"de6debd5a52fa5364aa888b6602d715ad6c8e478c4c3ce9d123c90ecb6d44236"} Jul 18 01:49:26.130325 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:26.130276 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-w98tz" podStartSLOduration=33.511103293 podStartE2EDuration="40.130257548s" totalImagesPullingTime="6.619154255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:26.12966582 +0000 UTC m=+74.982409165" watchObservedRunningTime="2026-07-18 01:49:26.130257548 +0000 UTC m=+74.983000893" Jul 18 01:49:27.114128 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:27.114085 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xmx42" event={"ID":"130558d4-120e-43cb-a644-e55e3321629b","Type":"ContainerStarted","Data":"f874566515fd86c510b7475f6c2563b488f8c7adfdffd0118e1d2a406ad8fa45"} Jul 18 01:49:27.116357 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:27.116327 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zl6kk" event={"ID":"5e4da607-7ffa-41ce-a806-b53359365b9a","Type":"ContainerStarted","Data":"8b679836650b4439483b6e3313b8a2b2a9000d7e828e2ff84c6d383616205f7a"} Jul 18 01:49:27.116512 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:27.116423 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zl6kk" Jul 18 01:49:27.118409 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:27.118360 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" event={"ID":"08596349-c720-4300-88ab-49a75b5e1f24","Type":"ContainerStarted","Data":"a249fbc176b339cfca99c9bbb144b3cf64f93f3c9a30a4aa75ce284fc28ffcd7"} Jul 18 01:49:27.134072 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:27.134025 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zl6kk" podStartSLOduration=34.121879316 podStartE2EDuration="41.134007384s" totalImagesPullingTime="7.012128068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:27.133401398 +0000 UTC m=+75.986144744" watchObservedRunningTime="2026-07-18 01:49:27.134007384 +0000 UTC m=+75.986750730" Jul 18 01:49:27.152507 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:27.152466 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7d4vr" podStartSLOduration=34.738595396 podStartE2EDuration="41.152451402s" totalImagesPullingTime="6.413856006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:27.15064094 +0000 UTC m=+76.003384285" watchObservedRunningTime="2026-07-18 01:49:27.152451402 +0000 UTC m=+76.005194747" Jul 18 01:49:27.170599 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:27.170555 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-525n9" podStartSLOduration=67.904894988 podStartE2EDuration="1m16.17053935s" totalImagesPullingTime="8.265644362s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:27.169409354 +0000 UTC m=+76.022152699" watchObservedRunningTime="2026-07-18 01:49:27.17053935 +0000 UTC m=+76.023282701" Jul 18 01:49:31.135445 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:31.135413 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xmx42" event={"ID":"130558d4-120e-43cb-a644-e55e3321629b","Type":"ContainerStarted","Data":"b62f1bf7aadaab411f5dc4aad48b001b4ae92a732e763ba217b3c9703965b7f4"} Jul 18 01:49:31.152461 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:31.152415 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xmx42" podStartSLOduration=17.71115436 podStartE2EDuration="35.152401585s" totalImagesPullingTime="17.441247225s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:48:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:31.151156954 +0000 UTC m=+80.003900298" watchObservedRunningTime="2026-07-18 01:49:31.152401585 +0000 UTC m=+80.005144931" Jul 18 01:49:31.544595 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:31.544571 2548 scope.go:117] "RemoveContainer" containerID="e69de986fb4725327a508cb371072f636047dc2df03776de7f7bd327828e7ad5" Jul 18 01:49:31.544756 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:31.544733 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-dhpvd_openshift-console-operator(7d9e6ac4-e8b2-40ef-88de-f70735119ce5)\"" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" podUID="7d9e6ac4-e8b2-40ef-88de-f70735119ce5" Jul 18 01:49:34.935736 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:34.935625 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ngj6b"] Jul 18 01:49:34.940221 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:34.940204 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:34.942992 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:34.942962 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 18 01:49:34.943117 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:34.943012 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 18 01:49:34.943117 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:34.943102 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 18 01:49:34.943912 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:34.943894 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6ncsh\"" Jul 18 01:49:34.944012 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:34.943928 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 18 01:49:35.043819 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.043799 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-metrics-client-ca\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.043928 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.043836 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qwxv\" (UniqueName: \"kubernetes.io/projected/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-kube-api-access-2qwxv\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.043928 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.043855 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.043928 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.043877 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-accelerators-collector-config\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.043928 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.043922 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-tls\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.044077 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.043945 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-wtmp\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.044077 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.043987 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-root\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.044077 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.044013 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-textfile\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.044077 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.044056 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-sys\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.144955 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.144933 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qwxv\" (UniqueName: \"kubernetes.io/projected/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-kube-api-access-2qwxv\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.145047 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.144958 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.145047 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.144982 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-accelerators-collector-config\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.145047 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.145001 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-tls\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.145047 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.145016 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-wtmp\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.145255 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.145060 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-root\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.145255 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.145113 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-textfile\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.145255 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:35.145134 2548 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 18 01:49:35.145255 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.145146 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-sys\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.145255 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.145162 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-wtmp\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.145255 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.145137 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-root\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.145255 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:49:35.145202 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-tls podName:9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc nodeName:}" failed. No retries permitted until 2026-07-18 01:49:35.64518122 +0000 UTC m=+84.497924556 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-tls") pod "node-exporter-ngj6b" (UID: "9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc") : secret "node-exporter-tls" not found Jul 18 01:49:35.145255 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.145216 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-sys\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.145255 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.145225 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-metrics-client-ca\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.145652 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.145466 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-textfile\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.145687 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.145669 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-accelerators-collector-config\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.145687 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.145675 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-metrics-client-ca\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.147302 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.147284 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.154886 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.154869 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qwxv\" (UniqueName: \"kubernetes.io/projected/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-kube-api-access-2qwxv\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.648232 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.648204 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-tls\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.650537 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.650514 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc-node-exporter-tls\") pod \"node-exporter-ngj6b\" (UID: \"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc\") " pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.849038 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:35.849012 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ngj6b" Jul 18 01:49:35.861961 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:49:35.861937 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a5e6c2b_2b9b_4ef1_ac2f_61eef2fac8bc.slice/crio-6f2a69783c7342f7c90d05c38c8b76fff3e46c31b131afc3941e2b6d3422f0cc WatchSource:0}: Error finding container 6f2a69783c7342f7c90d05c38c8b76fff3e46c31b131afc3941e2b6d3422f0cc: Status 404 returned error can't find the container with id 6f2a69783c7342f7c90d05c38c8b76fff3e46c31b131afc3941e2b6d3422f0cc Jul 18 01:49:36.156736 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:36.156687 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ngj6b" event={"ID":"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc","Type":"ContainerStarted","Data":"6f2a69783c7342f7c90d05c38c8b76fff3e46c31b131afc3941e2b6d3422f0cc"} Jul 18 01:49:37.125088 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:37.125005 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zl6kk" Jul 18 01:49:37.161246 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:37.161216 2548 generic.go:356] "Generic (PLEG): container finished" podID="9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc" containerID="f9591a91bcb7354806abdecbfbc13ec634eefd187ab72b584825628aa32fc4f6" exitCode=0 Jul 18 01:49:37.161624 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:37.161253 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ngj6b" event={"ID":"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc","Type":"ContainerDied","Data":"f9591a91bcb7354806abdecbfbc13ec634eefd187ab72b584825628aa32fc4f6"} Jul 18 01:49:38.170697 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:38.170660 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ngj6b" event={"ID":"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc","Type":"ContainerStarted","Data":"ce2432d2a2261a8fde571e4ea510d88c1acb641e0d9dc920f3322e37b8d20b57"} Jul 18 01:49:38.171076 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:38.170749 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ngj6b" event={"ID":"9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc","Type":"ContainerStarted","Data":"643adf8307faf555dc4a0bd28f22868bbdd4d2b8239ff8dc01aa942d16d74783"} Jul 18 01:49:38.190446 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:38.190402 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ngj6b" podStartSLOduration=3.302172945 podStartE2EDuration="4.190388897s" totalImagesPullingTime="825.635069ms" totalInitContainerRuntime="62.580883ms" isStatefulPod=true podCreationTimestamp="2026-07-18 01:49:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:38.188905176 +0000 UTC m=+87.041648520" watchObservedRunningTime="2026-07-18 01:49:38.190388897 +0000 UTC m=+87.043132264" Jul 18 01:49:38.620605 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:38.620515 2548 patch_prober.go:28] interesting pod/image-registry-b997b76d5-slg46 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 18 01:49:38.620605 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:38.620573 2548 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-b997b76d5-slg46" podUID="468d9d35-1c8b-404e-932d-39d32f02ea1b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 18 01:49:41.082470 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:41.082438 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:49:43.542986 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:43.542956 2548 scope.go:117] "RemoveContainer" containerID="e69de986fb4725327a508cb371072f636047dc2df03776de7f7bd327828e7ad5" Jul 18 01:49:44.192424 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:44.192396 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 01:49:44.192582 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:44.192497 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" event={"ID":"7d9e6ac4-e8b2-40ef-88de-f70735119ce5","Type":"ContainerStarted","Data":"d7cff057a266bec891c662d057d9c15c926a347a20830040df2815b86ba9ba9c"} Jul 18 01:49:44.192838 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:44.192808 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:49:44.209476 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:44.209431 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" podStartSLOduration=50.737360379 podStartE2EDuration="58.209416919s" totalImagesPullingTime="7.47205654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:48:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:49:44.20842406 +0000 UTC m=+93.061167405" watchObservedRunningTime="2026-07-18 01:49:44.209416919 +0000 UTC m=+93.062160263" Jul 18 01:49:44.374189 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:44.374156 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b997b76d5-slg46"] Jul 18 01:49:44.513760 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:44.513684 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-dhpvd" Jul 18 01:49:50.077239 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:49:50.077209 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cxfjn" Jul 18 01:50:09.394363 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.394323 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-b997b76d5-slg46" podUID="468d9d35-1c8b-404e-932d-39d32f02ea1b" containerName="registry" containerID="cri-o://4221d6c2e0df709b47f2e513964756ea3c918ce692864d27bb98f6ff870942fb" gracePeriod=30 Jul 18 01:50:09.638030 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.638008 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:50:09.684730 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.684704 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls\") pod \"468d9d35-1c8b-404e-932d-39d32f02ea1b\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " Jul 18 01:50:09.684863 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.684743 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wnfs2\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-kube-api-access-wnfs2\") pod \"468d9d35-1c8b-404e-932d-39d32f02ea1b\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " Jul 18 01:50:09.684863 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.684766 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/468d9d35-1c8b-404e-932d-39d32f02ea1b-image-registry-private-configuration\") pod \"468d9d35-1c8b-404e-932d-39d32f02ea1b\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " Jul 18 01:50:09.684952 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.684875 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/468d9d35-1c8b-404e-932d-39d32f02ea1b-ca-trust-extracted\") pod \"468d9d35-1c8b-404e-932d-39d32f02ea1b\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " Jul 18 01:50:09.684952 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.684901 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-certificates\") pod \"468d9d35-1c8b-404e-932d-39d32f02ea1b\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " Jul 18 01:50:09.684952 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.684932 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/468d9d35-1c8b-404e-932d-39d32f02ea1b-trusted-ca\") pod \"468d9d35-1c8b-404e-932d-39d32f02ea1b\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " Jul 18 01:50:09.685094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.684959 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-bound-sa-token\") pod \"468d9d35-1c8b-404e-932d-39d32f02ea1b\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " Jul 18 01:50:09.685094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.685005 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/468d9d35-1c8b-404e-932d-39d32f02ea1b-installation-pull-secrets\") pod \"468d9d35-1c8b-404e-932d-39d32f02ea1b\" (UID: \"468d9d35-1c8b-404e-932d-39d32f02ea1b\") " Jul 18 01:50:09.685396 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.685364 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468d9d35-1c8b-404e-932d-39d32f02ea1b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "468d9d35-1c8b-404e-932d-39d32f02ea1b" (UID: "468d9d35-1c8b-404e-932d-39d32f02ea1b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:50:09.685484 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.685375 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "468d9d35-1c8b-404e-932d-39d32f02ea1b" (UID: "468d9d35-1c8b-404e-932d-39d32f02ea1b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:50:09.687422 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.687389 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468d9d35-1c8b-404e-932d-39d32f02ea1b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "468d9d35-1c8b-404e-932d-39d32f02ea1b" (UID: "468d9d35-1c8b-404e-932d-39d32f02ea1b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:50:09.687533 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.687419 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "468d9d35-1c8b-404e-932d-39d32f02ea1b" (UID: "468d9d35-1c8b-404e-932d-39d32f02ea1b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:50:09.687533 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.687496 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "468d9d35-1c8b-404e-932d-39d32f02ea1b" (UID: "468d9d35-1c8b-404e-932d-39d32f02ea1b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:50:09.687615 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.687545 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-kube-api-access-wnfs2" (OuterVolumeSpecName: "kube-api-access-wnfs2") pod "468d9d35-1c8b-404e-932d-39d32f02ea1b" (UID: "468d9d35-1c8b-404e-932d-39d32f02ea1b"). InnerVolumeSpecName "kube-api-access-wnfs2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:50:09.687731 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.687709 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468d9d35-1c8b-404e-932d-39d32f02ea1b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "468d9d35-1c8b-404e-932d-39d32f02ea1b" (UID: "468d9d35-1c8b-404e-932d-39d32f02ea1b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:50:09.693877 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.693853 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/468d9d35-1c8b-404e-932d-39d32f02ea1b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "468d9d35-1c8b-404e-932d-39d32f02ea1b" (UID: "468d9d35-1c8b-404e-932d-39d32f02ea1b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:50:09.785905 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.785880 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wnfs2\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-kube-api-access-wnfs2\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:50:09.786008 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.785908 2548 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/468d9d35-1c8b-404e-932d-39d32f02ea1b-image-registry-private-configuration\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:50:09.786008 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.785923 2548 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/468d9d35-1c8b-404e-932d-39d32f02ea1b-ca-trust-extracted\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:50:09.786008 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.785934 2548 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-certificates\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:50:09.786008 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.785948 2548 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/468d9d35-1c8b-404e-932d-39d32f02ea1b-trusted-ca\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:50:09.786008 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.785960 2548 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-bound-sa-token\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:50:09.786008 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.785972 2548 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/468d9d35-1c8b-404e-932d-39d32f02ea1b-installation-pull-secrets\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:50:09.786008 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:09.785985 2548 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/468d9d35-1c8b-404e-932d-39d32f02ea1b-registry-tls\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:50:10.283149 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:10.283118 2548 generic.go:356] "Generic (PLEG): container finished" podID="468d9d35-1c8b-404e-932d-39d32f02ea1b" containerID="4221d6c2e0df709b47f2e513964756ea3c918ce692864d27bb98f6ff870942fb" exitCode=0 Jul 18 01:50:10.283342 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:10.283164 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b997b76d5-slg46" Jul 18 01:50:10.283342 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:10.283192 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b997b76d5-slg46" event={"ID":"468d9d35-1c8b-404e-932d-39d32f02ea1b","Type":"ContainerDied","Data":"4221d6c2e0df709b47f2e513964756ea3c918ce692864d27bb98f6ff870942fb"} Jul 18 01:50:10.283342 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:10.283240 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b997b76d5-slg46" event={"ID":"468d9d35-1c8b-404e-932d-39d32f02ea1b","Type":"ContainerDied","Data":"4ba2d0a33c914468f23dc31e8f72499e413242997b33c40769c2d0c3e0bfdaca"} Jul 18 01:50:10.283342 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:10.283266 2548 scope.go:117] "RemoveContainer" containerID="4221d6c2e0df709b47f2e513964756ea3c918ce692864d27bb98f6ff870942fb" Jul 18 01:50:10.291595 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:10.291579 2548 scope.go:117] "RemoveContainer" containerID="4221d6c2e0df709b47f2e513964756ea3c918ce692864d27bb98f6ff870942fb" Jul 18 01:50:10.291860 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:50:10.291840 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4221d6c2e0df709b47f2e513964756ea3c918ce692864d27bb98f6ff870942fb\": container with ID starting with 4221d6c2e0df709b47f2e513964756ea3c918ce692864d27bb98f6ff870942fb not found: ID does not exist" containerID="4221d6c2e0df709b47f2e513964756ea3c918ce692864d27bb98f6ff870942fb" Jul 18 01:50:10.291935 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:10.291869 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4221d6c2e0df709b47f2e513964756ea3c918ce692864d27bb98f6ff870942fb"} err="failed to get container status \"4221d6c2e0df709b47f2e513964756ea3c918ce692864d27bb98f6ff870942fb\": rpc error: code = NotFound desc = could not find container \"4221d6c2e0df709b47f2e513964756ea3c918ce692864d27bb98f6ff870942fb\": container with ID starting with 4221d6c2e0df709b47f2e513964756ea3c918ce692864d27bb98f6ff870942fb not found: ID does not exist" Jul 18 01:50:10.305120 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:10.305101 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b997b76d5-slg46"] Jul 18 01:50:10.310896 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:10.310876 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-b997b76d5-slg46"] Jul 18 01:50:11.546577 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:11.546541 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="468d9d35-1c8b-404e-932d-39d32f02ea1b" path="/var/lib/kubelet/pods/468d9d35-1c8b-404e-932d-39d32f02ea1b/volumes" Jul 18 01:50:16.309189 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:16.309157 2548 generic.go:356] "Generic (PLEG): container finished" podID="df23c9ba-7c94-4295-81a3-7452fb9ae1a9" containerID="8d64b1f06c05ae1662247260ad3c6e29cd06d793f11b2d7d71bf435f3b5a38ef" exitCode=0 Jul 18 01:50:16.309566 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:16.309211 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-8zxk6" event={"ID":"df23c9ba-7c94-4295-81a3-7452fb9ae1a9","Type":"ContainerDied","Data":"8d64b1f06c05ae1662247260ad3c6e29cd06d793f11b2d7d71bf435f3b5a38ef"} Jul 18 01:50:16.309566 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:16.309528 2548 scope.go:117] "RemoveContainer" containerID="8d64b1f06c05ae1662247260ad3c6e29cd06d793f11b2d7d71bf435f3b5a38ef" Jul 18 01:50:17.314224 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:17.314187 2548 generic.go:356] "Generic (PLEG): container finished" podID="2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1" containerID="66957fc81a1df43964ae1125a854a4c8bd253c795b59b7f1428468e3da3aea05" exitCode=0 Jul 18 01:50:17.314641 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:17.314260 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk" event={"ID":"2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1","Type":"ContainerDied","Data":"66957fc81a1df43964ae1125a854a4c8bd253c795b59b7f1428468e3da3aea05"} Jul 18 01:50:17.314641 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:17.314603 2548 scope.go:117] "RemoveContainer" containerID="66957fc81a1df43964ae1125a854a4c8bd253c795b59b7f1428468e3da3aea05" Jul 18 01:50:17.316074 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:17.316054 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-8zxk6" event={"ID":"df23c9ba-7c94-4295-81a3-7452fb9ae1a9","Type":"ContainerStarted","Data":"0fe77bc5c799831eb752a4e04631ddfb04999a6cc9871d4ff8382b91463ad41b"} Jul 18 01:50:18.321404 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:18.321373 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-64rzk" event={"ID":"2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1","Type":"ContainerStarted","Data":"02f13b6db691f8cdce926a992c8759fb7fd826501fdb1b8553fd92a6930317f6"} Jul 18 01:50:21.332878 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:21.332850 2548 generic.go:356] "Generic (PLEG): container finished" podID="31ce4668-2220-411e-929d-c6991e38932a" containerID="c7bbcb71d2d0bf545af06da08df0fe93a5eff8dba0d3143abc1f6d3dd2b29a69" exitCode=0 Jul 18 01:50:21.333240 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:21.332919 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s" event={"ID":"31ce4668-2220-411e-929d-c6991e38932a","Type":"ContainerDied","Data":"c7bbcb71d2d0bf545af06da08df0fe93a5eff8dba0d3143abc1f6d3dd2b29a69"} Jul 18 01:50:21.333240 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:21.333220 2548 scope.go:117] "RemoveContainer" containerID="c7bbcb71d2d0bf545af06da08df0fe93a5eff8dba0d3143abc1f6d3dd2b29a69" Jul 18 01:50:22.339966 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:22.339930 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-g9f9s" event={"ID":"31ce4668-2220-411e-929d-c6991e38932a","Type":"ContainerStarted","Data":"eb11ea7cbc946a383e3cee0712f49bfdfc975a248a32d373012803627bc6656d"} Jul 18 01:50:59.281579 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.281542 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8"] Jul 18 01:50:59.282096 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.281816 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="468d9d35-1c8b-404e-932d-39d32f02ea1b" containerName="registry" Jul 18 01:50:59.282096 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.281828 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="468d9d35-1c8b-404e-932d-39d32f02ea1b" containerName="registry" Jul 18 01:50:59.282096 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.281907 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="468d9d35-1c8b-404e-932d-39d32f02ea1b" containerName="registry" Jul 18 01:50:59.286091 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.286072 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.288679 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.288657 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 18 01:50:59.288679 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.288670 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 18 01:50:59.288876 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.288657 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 18 01:50:59.288876 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.288721 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 18 01:50:59.288876 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.288669 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-6vz9b\"" Jul 18 01:50:59.289202 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.289185 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 18 01:50:59.293103 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.292962 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 18 01:50:59.298147 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.298126 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8"] Jul 18 01:50:59.424422 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.424389 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c8d03cd-5411-4f24-825c-0ec7735d4672-metrics-client-ca\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.424422 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.424427 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c8d03cd-5411-4f24-825c-0ec7735d4672-secret-telemeter-client\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.424615 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.424451 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tk5x\" (UniqueName: \"kubernetes.io/projected/9c8d03cd-5411-4f24-825c-0ec7735d4672-kube-api-access-6tk5x\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.424615 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.424516 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c8d03cd-5411-4f24-825c-0ec7735d4672-telemeter-client-tls\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.424615 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.424581 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c8d03cd-5411-4f24-825c-0ec7735d4672-serving-certs-ca-bundle\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.424712 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.424633 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c8d03cd-5411-4f24-825c-0ec7735d4672-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.424712 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.424656 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c8d03cd-5411-4f24-825c-0ec7735d4672-federate-client-tls\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.424712 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.424683 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c8d03cd-5411-4f24-825c-0ec7735d4672-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.524981 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.524948 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c8d03cd-5411-4f24-825c-0ec7735d4672-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.524981 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.524986 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c8d03cd-5411-4f24-825c-0ec7735d4672-federate-client-tls\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.525240 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.525016 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c8d03cd-5411-4f24-825c-0ec7735d4672-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.525240 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.525142 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c8d03cd-5411-4f24-825c-0ec7735d4672-metrics-client-ca\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.525240 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.525190 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c8d03cd-5411-4f24-825c-0ec7735d4672-secret-telemeter-client\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.525240 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.525223 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tk5x\" (UniqueName: \"kubernetes.io/projected/9c8d03cd-5411-4f24-825c-0ec7735d4672-kube-api-access-6tk5x\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.525457 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.525249 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c8d03cd-5411-4f24-825c-0ec7735d4672-telemeter-client-tls\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.525457 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.525273 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c8d03cd-5411-4f24-825c-0ec7735d4672-serving-certs-ca-bundle\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.525907 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.525880 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9c8d03cd-5411-4f24-825c-0ec7735d4672-metrics-client-ca\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.526016 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.525964 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c8d03cd-5411-4f24-825c-0ec7735d4672-serving-certs-ca-bundle\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.526016 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.525994 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9c8d03cd-5411-4f24-825c-0ec7735d4672-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.527836 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.527808 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9c8d03cd-5411-4f24-825c-0ec7735d4672-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.527948 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.527931 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9c8d03cd-5411-4f24-825c-0ec7735d4672-secret-telemeter-client\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.527997 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.527934 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9c8d03cd-5411-4f24-825c-0ec7735d4672-federate-client-tls\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.529099 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.529084 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9c8d03cd-5411-4f24-825c-0ec7735d4672-telemeter-client-tls\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.533207 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.533150 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tk5x\" (UniqueName: \"kubernetes.io/projected/9c8d03cd-5411-4f24-825c-0ec7735d4672-kube-api-access-6tk5x\") pod \"telemeter-client-5f7bc69b77-cb2p8\" (UID: \"9c8d03cd-5411-4f24-825c-0ec7735d4672\") " pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.595327 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.595298 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" Jul 18 01:50:59.725302 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:50:59.725278 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8"] Jul 18 01:50:59.727528 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:50:59.727503 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c8d03cd_5411_4f24_825c_0ec7735d4672.slice/crio-6bdb44f2f6655fdaf775781b9daa3e4306476fdf777fe04c5d0d2ec012106e4c WatchSource:0}: Error finding container 6bdb44f2f6655fdaf775781b9daa3e4306476fdf777fe04c5d0d2ec012106e4c: Status 404 returned error can't find the container with id 6bdb44f2f6655fdaf775781b9daa3e4306476fdf777fe04c5d0d2ec012106e4c Jul 18 01:51:00.487795 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:51:00.487753 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" event={"ID":"9c8d03cd-5411-4f24-825c-0ec7735d4672","Type":"ContainerStarted","Data":"6bdb44f2f6655fdaf775781b9daa3e4306476fdf777fe04c5d0d2ec012106e4c"} Jul 18 01:51:02.497931 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:51:02.497897 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" event={"ID":"9c8d03cd-5411-4f24-825c-0ec7735d4672","Type":"ContainerStarted","Data":"9be44db407f9ecf1aaa74a61851ea6417052342ac0e8135b3d54922e76ccebac"} Jul 18 01:51:03.503075 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:51:03.502984 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" event={"ID":"9c8d03cd-5411-4f24-825c-0ec7735d4672","Type":"ContainerStarted","Data":"840400716bc0339f62322f5e15d9cd02cfe1acaa9f0897731e10a37057fe4d6e"} Jul 18 01:51:03.503075 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:51:03.503043 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" event={"ID":"9c8d03cd-5411-4f24-825c-0ec7735d4672","Type":"ContainerStarted","Data":"5a3b27bd64a047442e228b694f9e8d5ae5af17aecc4bbd31065056b7c8da1479"} Jul 18 01:51:03.524714 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:51:03.524666 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5f7bc69b77-cb2p8" podStartSLOduration=1.572030119 podStartE2EDuration="4.524649866s" totalImagesPullingTime="2.952619747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:50:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:51:03.522649355 +0000 UTC m=+172.375392699" watchObservedRunningTime="2026-07-18 01:51:03.524649866 +0000 UTC m=+172.377393209" Jul 18 01:52:48.112134 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.112045 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v"] Jul 18 01:52:48.115581 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.115558 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.117859 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.117834 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 18 01:52:48.118190 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.118170 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 18 01:52:48.118860 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.118838 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 18 01:52:48.118949 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.118864 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 18 01:52:48.118949 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.118847 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 18 01:52:48.118949 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.118890 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 18 01:52:48.119153 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.119138 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 18 01:52:48.125915 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.125895 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v"] Jul 18 01:52:48.143171 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.143150 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.143270 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.143177 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5np2g\" (UniqueName: \"kubernetes.io/projected/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-kube-api-access-5np2g\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.143270 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.143207 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-ca\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.143357 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.143275 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-hub\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.143357 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.143316 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.143357 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.143335 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.244524 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.244490 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5np2g\" (UniqueName: \"kubernetes.io/projected/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-kube-api-access-5np2g\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.244712 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.244536 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-ca\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.244712 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.244558 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-hub\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.244712 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.244586 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.244712 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.244682 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.244933 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.244800 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.245502 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.245473 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-additional-service-ca\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.247373 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.247344 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-ca\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.247485 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.247379 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.247485 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.247385 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-hub\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.247485 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.247385 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.252332 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.252311 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5np2g\" (UniqueName: \"kubernetes.io/projected/2c8245bb-beb2-4608-bb00-f37ab8a3b31f-kube-api-access-5np2g\") pod \"cluster-proxy-proxy-agent-cd8dffc7c-pgd5v\" (UID: \"2c8245bb-beb2-4608-bb00-f37ab8a3b31f\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.431935 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.431851 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" Jul 18 01:52:48.565832 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.563079 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v"] Jul 18 01:52:48.569644 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:52:48.569615 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c8245bb_beb2_4608_bb00_f37ab8a3b31f.slice/crio-f55436345de836da5a51b978fe1f157c62ceeefda60303a99cbb3ffd54df2277 WatchSource:0}: Error finding container f55436345de836da5a51b978fe1f157c62ceeefda60303a99cbb3ffd54df2277: Status 404 returned error can't find the container with id f55436345de836da5a51b978fe1f157c62ceeefda60303a99cbb3ffd54df2277 Jul 18 01:52:48.875350 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:48.875261 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" event={"ID":"2c8245bb-beb2-4608-bb00-f37ab8a3b31f","Type":"ContainerStarted","Data":"f55436345de836da5a51b978fe1f157c62ceeefda60303a99cbb3ffd54df2277"} Jul 18 01:52:52.894196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:52.894156 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" event={"ID":"2c8245bb-beb2-4608-bb00-f37ab8a3b31f","Type":"ContainerStarted","Data":"eab3c6fa8ba6673b69e98d01911c1ffa8b97d570253ba6c6748f405c7d521351"} Jul 18 01:52:52.894616 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:52.894209 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" event={"ID":"2c8245bb-beb2-4608-bb00-f37ab8a3b31f","Type":"ContainerStarted","Data":"3b3032873e7f9f43efdd25cf3639a6ad60f4a904f1da8121e52ee904792e73b8"} Jul 18 01:52:52.894616 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:52.894228 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" event={"ID":"2c8245bb-beb2-4608-bb00-f37ab8a3b31f","Type":"ContainerStarted","Data":"656934d64ef39a7a546e88e2fd925d98699b74abcd49d7b4733ab3215c0b8621"} Jul 18 01:52:52.917413 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:52:52.917360 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-cd8dffc7c-pgd5v" podStartSLOduration=1.537643586 podStartE2EDuration="4.917343572s" totalImagesPullingTime="3.379699986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:52:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:52:52.915259241 +0000 UTC m=+281.768002586" watchObservedRunningTime="2026-07-18 01:52:52.917343572 +0000 UTC m=+281.770086916" Jul 18 01:53:11.446918 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:11.446884 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 01:53:11.449668 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:11.449643 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 01:53:11.459035 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:11.459011 2548 kubelet.go:1648] "Image garbage collection succeeded" Jul 18 01:53:59.250145 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.250114 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76"] Jul 18 01:53:59.253365 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.253348 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:53:59.261926 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.261903 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 18 01:53:59.263121 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.263102 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 18 01:53:59.264164 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.264146 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 18 01:53:59.264273 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.264152 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 18 01:53:59.265235 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.265220 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-hw5p9\"" Jul 18 01:53:59.268721 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.268699 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a9da053-d6e0-4e23-bbf0-1765e1ded023-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-6wk76\" (UID: \"6a9da053-d6e0-4e23-bbf0-1765e1ded023\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:53:59.268821 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.268739 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a9da053-d6e0-4e23-bbf0-1765e1ded023-tmp\") pod \"opendatahub-operator-controller-manager-84c588c4c7-6wk76\" (UID: \"6a9da053-d6e0-4e23-bbf0-1765e1ded023\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:53:59.268821 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.268807 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdlcj\" (UniqueName: \"kubernetes.io/projected/6a9da053-d6e0-4e23-bbf0-1765e1ded023-kube-api-access-pdlcj\") pod \"opendatahub-operator-controller-manager-84c588c4c7-6wk76\" (UID: \"6a9da053-d6e0-4e23-bbf0-1765e1ded023\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:53:59.268892 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.268839 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6a9da053-d6e0-4e23-bbf0-1765e1ded023-manifests\") pod \"opendatahub-operator-controller-manager-84c588c4c7-6wk76\" (UID: \"6a9da053-d6e0-4e23-bbf0-1765e1ded023\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:53:59.268892 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.268876 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a9da053-d6e0-4e23-bbf0-1765e1ded023-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-6wk76\" (UID: \"6a9da053-d6e0-4e23-bbf0-1765e1ded023\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:53:59.294446 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.294426 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76"] Jul 18 01:53:59.369758 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.369733 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a9da053-d6e0-4e23-bbf0-1765e1ded023-tmp\") pod \"opendatahub-operator-controller-manager-84c588c4c7-6wk76\" (UID: \"6a9da053-d6e0-4e23-bbf0-1765e1ded023\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:53:59.369877 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.369770 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdlcj\" (UniqueName: \"kubernetes.io/projected/6a9da053-d6e0-4e23-bbf0-1765e1ded023-kube-api-access-pdlcj\") pod \"opendatahub-operator-controller-manager-84c588c4c7-6wk76\" (UID: \"6a9da053-d6e0-4e23-bbf0-1765e1ded023\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:53:59.369877 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.369801 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6a9da053-d6e0-4e23-bbf0-1765e1ded023-manifests\") pod \"opendatahub-operator-controller-manager-84c588c4c7-6wk76\" (UID: \"6a9da053-d6e0-4e23-bbf0-1765e1ded023\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:53:59.369877 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.369836 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a9da053-d6e0-4e23-bbf0-1765e1ded023-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-6wk76\" (UID: \"6a9da053-d6e0-4e23-bbf0-1765e1ded023\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:53:59.369877 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.369856 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a9da053-d6e0-4e23-bbf0-1765e1ded023-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-6wk76\" (UID: \"6a9da053-d6e0-4e23-bbf0-1765e1ded023\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:53:59.370230 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.370211 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/6a9da053-d6e0-4e23-bbf0-1765e1ded023-manifests\") pod \"opendatahub-operator-controller-manager-84c588c4c7-6wk76\" (UID: \"6a9da053-d6e0-4e23-bbf0-1765e1ded023\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:53:59.371910 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.371890 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a9da053-d6e0-4e23-bbf0-1765e1ded023-tmp\") pod \"opendatahub-operator-controller-manager-84c588c4c7-6wk76\" (UID: \"6a9da053-d6e0-4e23-bbf0-1765e1ded023\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:53:59.372202 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.372184 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/6a9da053-d6e0-4e23-bbf0-1765e1ded023-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-6wk76\" (UID: \"6a9da053-d6e0-4e23-bbf0-1765e1ded023\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:53:59.372247 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.372227 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6a9da053-d6e0-4e23-bbf0-1765e1ded023-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c588c4c7-6wk76\" (UID: \"6a9da053-d6e0-4e23-bbf0-1765e1ded023\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:53:59.389142 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.389117 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdlcj\" (UniqueName: \"kubernetes.io/projected/6a9da053-d6e0-4e23-bbf0-1765e1ded023-kube-api-access-pdlcj\") pod \"opendatahub-operator-controller-manager-84c588c4c7-6wk76\" (UID: \"6a9da053-d6e0-4e23-bbf0-1765e1ded023\") " pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:53:59.563545 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.563459 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:53:59.701186 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.701161 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76"] Jul 18 01:53:59.704011 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:53:59.703980 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a9da053_d6e0_4e23_bbf0_1765e1ded023.slice/crio-fba9574ea488481824ccfa13e17c07d1e5677380230cb217984757dcb4dc16b8 WatchSource:0}: Error finding container fba9574ea488481824ccfa13e17c07d1e5677380230cb217984757dcb4dc16b8: Status 404 returned error can't find the container with id fba9574ea488481824ccfa13e17c07d1e5677380230cb217984757dcb4dc16b8 Jul 18 01:53:59.705669 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:53:59.705651 2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 01:54:00.132407 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:00.132372 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" event={"ID":"6a9da053-d6e0-4e23-bbf0-1765e1ded023","Type":"ContainerStarted","Data":"fba9574ea488481824ccfa13e17c07d1e5677380230cb217984757dcb4dc16b8"} Jul 18 01:54:03.149331 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:03.149288 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" event={"ID":"6a9da053-d6e0-4e23-bbf0-1765e1ded023","Type":"ContainerStarted","Data":"645b71f5861da259a07b170096bc4ded8efb7e4ac38073cb5cfd036fd8ead50d"} Jul 18 01:54:04.155867 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:04.155837 2548 generic.go:356] "Generic (PLEG): container finished" podID="6a9da053-d6e0-4e23-bbf0-1765e1ded023" containerID="645b71f5861da259a07b170096bc4ded8efb7e4ac38073cb5cfd036fd8ead50d" exitCode=0 Jul 18 01:54:04.156206 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:04.155897 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" event={"ID":"6a9da053-d6e0-4e23-bbf0-1765e1ded023","Type":"ContainerDied","Data":"645b71f5861da259a07b170096bc4ded8efb7e4ac38073cb5cfd036fd8ead50d"} Jul 18 01:54:05.161600 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:05.161564 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" event={"ID":"6a9da053-d6e0-4e23-bbf0-1765e1ded023","Type":"ContainerStarted","Data":"121fe715d72bfa8e6434be4aa8009af512251a39e22037a22ab34ac6fffae3a9"} Jul 18 01:54:05.162002 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:05.161710 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:54:05.195568 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:05.195529 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" podStartSLOduration=1.691118619 podStartE2EDuration="6.195513802s" totalImagesPullingTime="3.395866356s" totalInitContainerRuntime="1.108528827s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:53:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:05.19537256 +0000 UTC m=+354.048115904" watchObservedRunningTime="2026-07-18 01:54:05.195513802 +0000 UTC m=+354.048257145" Jul 18 01:54:16.168334 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:16.168305 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84c588c4c7-6wk76" Jul 18 01:54:19.345793 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.345755 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c65d4b6f-g5q5h"] Jul 18 01:54:19.358640 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.358615 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-g5q5h" Jul 18 01:54:19.361059 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.361036 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 18 01:54:19.361293 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.361273 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 18 01:54:19.361395 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.361370 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9ws6j\"" Jul 18 01:54:19.367938 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.367911 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c65d4b6f-g5q5h"] Jul 18 01:54:19.412529 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.412506 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56701e43-78f7-4acb-8f31-b3f75fc147af-tmp\") pod \"kube-auth-proxy-7c65d4b6f-g5q5h\" (UID: \"56701e43-78f7-4acb-8f31-b3f75fc147af\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-g5q5h" Jul 18 01:54:19.412666 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.412544 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxxfd\" (UniqueName: \"kubernetes.io/projected/56701e43-78f7-4acb-8f31-b3f75fc147af-kube-api-access-lxxfd\") pod \"kube-auth-proxy-7c65d4b6f-g5q5h\" (UID: \"56701e43-78f7-4acb-8f31-b3f75fc147af\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-g5q5h" Jul 18 01:54:19.412666 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.412564 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56701e43-78f7-4acb-8f31-b3f75fc147af-tls-certs\") pod \"kube-auth-proxy-7c65d4b6f-g5q5h\" (UID: \"56701e43-78f7-4acb-8f31-b3f75fc147af\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-g5q5h" Jul 18 01:54:19.513971 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.513931 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56701e43-78f7-4acb-8f31-b3f75fc147af-tmp\") pod \"kube-auth-proxy-7c65d4b6f-g5q5h\" (UID: \"56701e43-78f7-4acb-8f31-b3f75fc147af\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-g5q5h" Jul 18 01:54:19.514128 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.513993 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxxfd\" (UniqueName: \"kubernetes.io/projected/56701e43-78f7-4acb-8f31-b3f75fc147af-kube-api-access-lxxfd\") pod \"kube-auth-proxy-7c65d4b6f-g5q5h\" (UID: \"56701e43-78f7-4acb-8f31-b3f75fc147af\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-g5q5h" Jul 18 01:54:19.514128 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.514015 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56701e43-78f7-4acb-8f31-b3f75fc147af-tls-certs\") pod \"kube-auth-proxy-7c65d4b6f-g5q5h\" (UID: \"56701e43-78f7-4acb-8f31-b3f75fc147af\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-g5q5h" Jul 18 01:54:19.516273 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.516213 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56701e43-78f7-4acb-8f31-b3f75fc147af-tmp\") pod \"kube-auth-proxy-7c65d4b6f-g5q5h\" (UID: \"56701e43-78f7-4acb-8f31-b3f75fc147af\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-g5q5h" Jul 18 01:54:19.516488 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.516468 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56701e43-78f7-4acb-8f31-b3f75fc147af-tls-certs\") pod \"kube-auth-proxy-7c65d4b6f-g5q5h\" (UID: \"56701e43-78f7-4acb-8f31-b3f75fc147af\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-g5q5h" Jul 18 01:54:19.530120 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.530098 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxxfd\" (UniqueName: \"kubernetes.io/projected/56701e43-78f7-4acb-8f31-b3f75fc147af-kube-api-access-lxxfd\") pod \"kube-auth-proxy-7c65d4b6f-g5q5h\" (UID: \"56701e43-78f7-4acb-8f31-b3f75fc147af\") " pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-g5q5h" Jul 18 01:54:19.561247 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.561219 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g57qp"] Jul 18 01:54:19.565020 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.565003 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-g57qp" Jul 18 01:54:19.567336 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.567310 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 18 01:54:19.567463 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.567443 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-k45d8\"" Jul 18 01:54:19.583057 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.583032 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g57qp"] Jul 18 01:54:19.614628 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.614534 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/953b02cb-cd60-4aa0-b742-e5035254d741-cert\") pod \"odh-model-controller-7998c67dc4-g57qp\" (UID: \"953b02cb-cd60-4aa0-b742-e5035254d741\") " pod="opendatahub/odh-model-controller-7998c67dc4-g57qp" Jul 18 01:54:19.614628 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.614586 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkcpw\" (UniqueName: \"kubernetes.io/projected/953b02cb-cd60-4aa0-b742-e5035254d741-kube-api-access-rkcpw\") pod \"odh-model-controller-7998c67dc4-g57qp\" (UID: \"953b02cb-cd60-4aa0-b742-e5035254d741\") " pod="opendatahub/odh-model-controller-7998c67dc4-g57qp" Jul 18 01:54:19.669661 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.669632 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-g5q5h" Jul 18 01:54:19.716010 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.715973 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/953b02cb-cd60-4aa0-b742-e5035254d741-cert\") pod \"odh-model-controller-7998c67dc4-g57qp\" (UID: \"953b02cb-cd60-4aa0-b742-e5035254d741\") " pod="opendatahub/odh-model-controller-7998c67dc4-g57qp" Jul 18 01:54:19.716179 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.716021 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkcpw\" (UniqueName: \"kubernetes.io/projected/953b02cb-cd60-4aa0-b742-e5035254d741-kube-api-access-rkcpw\") pod \"odh-model-controller-7998c67dc4-g57qp\" (UID: \"953b02cb-cd60-4aa0-b742-e5035254d741\") " pod="opendatahub/odh-model-controller-7998c67dc4-g57qp" Jul 18 01:54:19.716253 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:54:19.716191 2548 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 18 01:54:19.716305 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:54:19.716261 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/953b02cb-cd60-4aa0-b742-e5035254d741-cert podName:953b02cb-cd60-4aa0-b742-e5035254d741 nodeName:}" failed. No retries permitted until 2026-07-18 01:54:20.21623976 +0000 UTC m=+369.068983097 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/953b02cb-cd60-4aa0-b742-e5035254d741-cert") pod "odh-model-controller-7998c67dc4-g57qp" (UID: "953b02cb-cd60-4aa0-b742-e5035254d741") : secret "odh-model-controller-webhook-cert" not found Jul 18 01:54:19.726305 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.726279 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkcpw\" (UniqueName: \"kubernetes.io/projected/953b02cb-cd60-4aa0-b742-e5035254d741-kube-api-access-rkcpw\") pod \"odh-model-controller-7998c67dc4-g57qp\" (UID: \"953b02cb-cd60-4aa0-b742-e5035254d741\") " pod="opendatahub/odh-model-controller-7998c67dc4-g57qp" Jul 18 01:54:19.808792 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:19.808757 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c65d4b6f-g5q5h"] Jul 18 01:54:19.811122 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:54:19.811097 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56701e43_78f7_4acb_8f31_b3f75fc147af.slice/crio-fd96c5c586402d216a3a9de12b3fd48f320b9bef328dc805b2de2450ba9e793b WatchSource:0}: Error finding container fd96c5c586402d216a3a9de12b3fd48f320b9bef328dc805b2de2450ba9e793b: Status 404 returned error can't find the container with id fd96c5c586402d216a3a9de12b3fd48f320b9bef328dc805b2de2450ba9e793b Jul 18 01:54:20.219424 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.219382 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/953b02cb-cd60-4aa0-b742-e5035254d741-cert\") pod \"odh-model-controller-7998c67dc4-g57qp\" (UID: \"953b02cb-cd60-4aa0-b742-e5035254d741\") " pod="opendatahub/odh-model-controller-7998c67dc4-g57qp" Jul 18 01:54:20.221896 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.221873 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/953b02cb-cd60-4aa0-b742-e5035254d741-cert\") pod \"odh-model-controller-7998c67dc4-g57qp\" (UID: \"953b02cb-cd60-4aa0-b742-e5035254d741\") " pod="opendatahub/odh-model-controller-7998c67dc4-g57qp" Jul 18 01:54:20.224055 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.224030 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-g5q5h" event={"ID":"56701e43-78f7-4acb-8f31-b3f75fc147af","Type":"ContainerStarted","Data":"fd96c5c586402d216a3a9de12b3fd48f320b9bef328dc805b2de2450ba9e793b"} Jul 18 01:54:20.475603 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.475510 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-g57qp" Jul 18 01:54:20.651499 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.651461 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-g57qp"] Jul 18 01:54:20.654693 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:54:20.654667 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod953b02cb_cd60_4aa0_b742_e5035254d741.slice/crio-b5f162254bbdd8bbf7d6acca0ea8c3fe3fd688d7297aab25bc27a64cf0bd4464 WatchSource:0}: Error finding container b5f162254bbdd8bbf7d6acca0ea8c3fe3fd688d7297aab25bc27a64cf0bd4464: Status 404 returned error can't find the container with id b5f162254bbdd8bbf7d6acca0ea8c3fe3fd688d7297aab25bc27a64cf0bd4464 Jul 18 01:54:20.661141 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.661118 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bk8lx"] Jul 18 01:54:20.672917 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.672883 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-bk8lx" Jul 18 01:54:20.673702 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.673677 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bk8lx"] Jul 18 01:54:20.676158 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.675962 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 18 01:54:20.725156 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.725127 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrxrh\" (UniqueName: \"kubernetes.io/projected/bca4080e-2f9e-453e-8d6b-6179b8665cfd-kube-api-access-rrxrh\") pod \"model-serving-api-54d89bd5fc-bk8lx\" (UID: \"bca4080e-2f9e-453e-8d6b-6179b8665cfd\") " pod="opendatahub/model-serving-api-54d89bd5fc-bk8lx" Jul 18 01:54:20.725314 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.725197 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bca4080e-2f9e-453e-8d6b-6179b8665cfd-tls-certs\") pod \"model-serving-api-54d89bd5fc-bk8lx\" (UID: \"bca4080e-2f9e-453e-8d6b-6179b8665cfd\") " pod="opendatahub/model-serving-api-54d89bd5fc-bk8lx" Jul 18 01:54:20.826338 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.826259 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bca4080e-2f9e-453e-8d6b-6179b8665cfd-tls-certs\") pod \"model-serving-api-54d89bd5fc-bk8lx\" (UID: \"bca4080e-2f9e-453e-8d6b-6179b8665cfd\") " pod="opendatahub/model-serving-api-54d89bd5fc-bk8lx" Jul 18 01:54:20.826485 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.826361 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrxrh\" (UniqueName: \"kubernetes.io/projected/bca4080e-2f9e-453e-8d6b-6179b8665cfd-kube-api-access-rrxrh\") pod \"model-serving-api-54d89bd5fc-bk8lx\" (UID: \"bca4080e-2f9e-453e-8d6b-6179b8665cfd\") " pod="opendatahub/model-serving-api-54d89bd5fc-bk8lx" Jul 18 01:54:20.829552 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.829523 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bca4080e-2f9e-453e-8d6b-6179b8665cfd-tls-certs\") pod \"model-serving-api-54d89bd5fc-bk8lx\" (UID: \"bca4080e-2f9e-453e-8d6b-6179b8665cfd\") " pod="opendatahub/model-serving-api-54d89bd5fc-bk8lx" Jul 18 01:54:20.846372 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.846341 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrxrh\" (UniqueName: \"kubernetes.io/projected/bca4080e-2f9e-453e-8d6b-6179b8665cfd-kube-api-access-rrxrh\") pod \"model-serving-api-54d89bd5fc-bk8lx\" (UID: \"bca4080e-2f9e-453e-8d6b-6179b8665cfd\") " pod="opendatahub/model-serving-api-54d89bd5fc-bk8lx" Jul 18 01:54:20.850834 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.850806 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc"] Jul 18 01:54:20.854820 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.854738 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:20.857757 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.857522 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 18 01:54:20.857757 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.857550 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 18 01:54:20.857757 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.857616 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-r9btw\"" Jul 18 01:54:20.857989 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.857920 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 18 01:54:20.858231 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.858137 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 18 01:54:20.871899 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.871873 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc"] Jul 18 01:54:20.927486 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.927458 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:20.927656 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.927491 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-token\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:20.927656 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.927548 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-cacerts\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:20.927656 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.927580 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/76e66dae-0b7d-4e97-9341-0eebd53114f8-local-certs\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:20.927656 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.927604 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:20.927656 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.927645 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:20.927957 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.927669 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzn97\" (UniqueName: \"kubernetes.io/projected/76e66dae-0b7d-4e97-9341-0eebd53114f8-kube-api-access-xzn97\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:20.988207 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:20.988172 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-bk8lx" Jul 18 01:54:21.028920 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.028886 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-cacerts\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:21.029034 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.028937 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/76e66dae-0b7d-4e97-9341-0eebd53114f8-local-certs\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:21.029034 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.028966 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:21.029034 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.028994 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:21.029034 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.029022 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzn97\" (UniqueName: \"kubernetes.io/projected/76e66dae-0b7d-4e97-9341-0eebd53114f8-kube-api-access-xzn97\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:21.029169 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.029095 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:21.029169 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.029120 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-token\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:21.030261 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.030213 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:21.031715 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.031688 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-cacerts\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:21.032150 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.032130 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/76e66dae-0b7d-4e97-9341-0eebd53114f8-local-certs\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:21.032222 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.032197 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:21.032669 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.032649 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:21.038613 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.038592 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-token\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:21.039858 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.039834 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzn97\" (UniqueName: \"kubernetes.io/projected/76e66dae-0b7d-4e97-9341-0eebd53114f8-kube-api-access-xzn97\") pod \"istiod-openshift-gateway-7574459656-5rgbc\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:21.147568 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.147489 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bk8lx"] Jul 18 01:54:21.150533 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:54:21.150495 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbca4080e_2f9e_453e_8d6b_6179b8665cfd.slice/crio-362f26ac35c5ab4a207623b3d669c08470c2b0a105eb3270bd277dde1676cdd4 WatchSource:0}: Error finding container 362f26ac35c5ab4a207623b3d669c08470c2b0a105eb3270bd277dde1676cdd4: Status 404 returned error can't find the container with id 362f26ac35c5ab4a207623b3d669c08470c2b0a105eb3270bd277dde1676cdd4 Jul 18 01:54:21.167124 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.167095 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:21.231458 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.231418 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-g57qp" event={"ID":"953b02cb-cd60-4aa0-b742-e5035254d741","Type":"ContainerStarted","Data":"b5f162254bbdd8bbf7d6acca0ea8c3fe3fd688d7297aab25bc27a64cf0bd4464"} Jul 18 01:54:21.237823 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.237534 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-bk8lx" event={"ID":"bca4080e-2f9e-453e-8d6b-6179b8665cfd","Type":"ContainerStarted","Data":"362f26ac35c5ab4a207623b3d669c08470c2b0a105eb3270bd277dde1676cdd4"} Jul 18 01:54:21.346645 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:21.346413 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc"] Jul 18 01:54:21.352728 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:54:21.352622 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76e66dae_0b7d_4e97_9341_0eebd53114f8.slice/crio-15a13d2903cf7a4b9dd0efd1fb7a7e5d4644188d5d603a47a51dd319ebdd761b WatchSource:0}: Error finding container 15a13d2903cf7a4b9dd0efd1fb7a7e5d4644188d5d603a47a51dd319ebdd761b: Status 404 returned error can't find the container with id 15a13d2903cf7a4b9dd0efd1fb7a7e5d4644188d5d603a47a51dd319ebdd761b Jul 18 01:54:22.243857 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:22.243811 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" event={"ID":"76e66dae-0b7d-4e97-9341-0eebd53114f8","Type":"ContainerStarted","Data":"15a13d2903cf7a4b9dd0efd1fb7a7e5d4644188d5d603a47a51dd319ebdd761b"} Jul 18 01:54:25.270857 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:25.270815 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-g5q5h" event={"ID":"56701e43-78f7-4acb-8f31-b3f75fc147af","Type":"ContainerStarted","Data":"0d6f9d484db2e4f06e361f1e1a5c2d619b8286b90cce346a6701bad6151bbbe2"} Jul 18 01:54:25.288547 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:25.288502 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c65d4b6f-g5q5h" podStartSLOduration=2.18277053 podStartE2EDuration="6.288485014s" totalImagesPullingTime="4.105714484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:25.28708579 +0000 UTC m=+374.139829135" watchObservedRunningTime="2026-07-18 01:54:25.288485014 +0000 UTC m=+374.141228360" Jul 18 01:54:25.494248 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:25.494215 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:54:25.494361 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:25.494277 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:54:26.277891 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:26.277840 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-g57qp" event={"ID":"953b02cb-cd60-4aa0-b742-e5035254d741","Type":"ContainerStarted","Data":"a5183b61518391cb683263aeda691f07b18bac4685c6fe7b402ef596807b19ab"} Jul 18 01:54:26.278349 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:26.277909 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-g57qp" Jul 18 01:54:26.279554 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:26.279526 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-bk8lx" event={"ID":"bca4080e-2f9e-453e-8d6b-6179b8665cfd","Type":"ContainerStarted","Data":"ce4169593ce6bbcaf8536aacdd77d83d176e53cb4b0f710340c5ca65cfd0f12a"} Jul 18 01:54:26.279686 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:26.279632 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-bk8lx" Jul 18 01:54:26.281125 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:26.281093 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" event={"ID":"76e66dae-0b7d-4e97-9341-0eebd53114f8","Type":"ContainerStarted","Data":"5ddea3c3d21a62107354e01c3e372d37d9867b8b5f2b757755fe421ce5de0771"} Jul 18 01:54:26.295185 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:26.295145 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-g57qp" podStartSLOduration=2.519971956 podStartE2EDuration="7.295130124s" totalImagesPullingTime="4.775158168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:26.294061631 +0000 UTC m=+375.146804977" watchObservedRunningTime="2026-07-18 01:54:26.295130124 +0000 UTC m=+375.147873468" Jul 18 01:54:26.314959 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:26.314921 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" podStartSLOduration=2.176668312 podStartE2EDuration="6.314907104s" totalImagesPullingTime="4.138238792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:26.312968907 +0000 UTC m=+375.165712252" watchObservedRunningTime="2026-07-18 01:54:26.314907104 +0000 UTC m=+375.167650448" Jul 18 01:54:26.329380 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:26.329344 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-bk8lx" podStartSLOduration=2.04384542 podStartE2EDuration="6.329331044s" totalImagesPullingTime="4.285485624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:26.327839684 +0000 UTC m=+375.180583019" watchObservedRunningTime="2026-07-18 01:54:26.329331044 +0000 UTC m=+375.182074389" Jul 18 01:54:27.286574 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:27.286536 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:27.288194 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:27.288166 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:30.929381 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:30.929349 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pvq8t"] Jul 18 01:54:30.932371 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:30.932353 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pvq8t" Jul 18 01:54:30.936472 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:30.936447 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 18 01:54:30.936719 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:30.936703 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-27rt6\"" Jul 18 01:54:30.937253 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:30.937233 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-b8nbn"] Jul 18 01:54:30.940347 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:30.940324 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b8nbn" Jul 18 01:54:30.943059 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:30.943041 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 18 01:54:30.943824 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:30.943804 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-scxst\"" Jul 18 01:54:30.946255 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:30.946231 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pvq8t"] Jul 18 01:54:30.953435 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:30.953415 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-b8nbn"] Jul 18 01:54:31.026526 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:31.026499 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbnfp\" (UniqueName: \"kubernetes.io/projected/c0e8558a-a0ba-4d9c-a4bc-4cab55db2a77-kube-api-access-hbnfp\") pod \"llmisvc-controller-manager-777c6bd7d5-pvq8t\" (UID: \"c0e8558a-a0ba-4d9c-a4bc-4cab55db2a77\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pvq8t" Jul 18 01:54:31.026676 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:31.026608 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a20675e-03ef-44a5-8663-ead09b5e6774-cert\") pod \"kserve-controller-manager-d6f4b65b8-b8nbn\" (UID: \"8a20675e-03ef-44a5-8663-ead09b5e6774\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b8nbn" Jul 18 01:54:31.026676 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:31.026628 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0e8558a-a0ba-4d9c-a4bc-4cab55db2a77-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pvq8t\" (UID: \"c0e8558a-a0ba-4d9c-a4bc-4cab55db2a77\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pvq8t" Jul 18 01:54:31.026676 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:31.026652 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzkql\" (UniqueName: \"kubernetes.io/projected/8a20675e-03ef-44a5-8663-ead09b5e6774-kube-api-access-kzkql\") pod \"kserve-controller-manager-d6f4b65b8-b8nbn\" (UID: \"8a20675e-03ef-44a5-8663-ead09b5e6774\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b8nbn" Jul 18 01:54:31.127085 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:31.127062 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0e8558a-a0ba-4d9c-a4bc-4cab55db2a77-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pvq8t\" (UID: \"c0e8558a-a0ba-4d9c-a4bc-4cab55db2a77\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pvq8t" Jul 18 01:54:31.127236 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:31.127095 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzkql\" (UniqueName: \"kubernetes.io/projected/8a20675e-03ef-44a5-8663-ead09b5e6774-kube-api-access-kzkql\") pod \"kserve-controller-manager-d6f4b65b8-b8nbn\" (UID: \"8a20675e-03ef-44a5-8663-ead09b5e6774\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b8nbn" Jul 18 01:54:31.127236 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:31.127123 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbnfp\" (UniqueName: \"kubernetes.io/projected/c0e8558a-a0ba-4d9c-a4bc-4cab55db2a77-kube-api-access-hbnfp\") pod \"llmisvc-controller-manager-777c6bd7d5-pvq8t\" (UID: \"c0e8558a-a0ba-4d9c-a4bc-4cab55db2a77\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pvq8t" Jul 18 01:54:31.127236 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:31.127180 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a20675e-03ef-44a5-8663-ead09b5e6774-cert\") pod \"kserve-controller-manager-d6f4b65b8-b8nbn\" (UID: \"8a20675e-03ef-44a5-8663-ead09b5e6774\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b8nbn" Jul 18 01:54:31.129486 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:31.129468 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c0e8558a-a0ba-4d9c-a4bc-4cab55db2a77-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-pvq8t\" (UID: \"c0e8558a-a0ba-4d9c-a4bc-4cab55db2a77\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pvq8t" Jul 18 01:54:31.129486 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:31.129483 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a20675e-03ef-44a5-8663-ead09b5e6774-cert\") pod \"kserve-controller-manager-d6f4b65b8-b8nbn\" (UID: \"8a20675e-03ef-44a5-8663-ead09b5e6774\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b8nbn" Jul 18 01:54:31.146801 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:31.146759 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbnfp\" (UniqueName: \"kubernetes.io/projected/c0e8558a-a0ba-4d9c-a4bc-4cab55db2a77-kube-api-access-hbnfp\") pod \"llmisvc-controller-manager-777c6bd7d5-pvq8t\" (UID: \"c0e8558a-a0ba-4d9c-a4bc-4cab55db2a77\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pvq8t" Jul 18 01:54:31.148574 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:31.148554 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzkql\" (UniqueName: \"kubernetes.io/projected/8a20675e-03ef-44a5-8663-ead09b5e6774-kube-api-access-kzkql\") pod \"kserve-controller-manager-d6f4b65b8-b8nbn\" (UID: \"8a20675e-03ef-44a5-8663-ead09b5e6774\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-b8nbn" Jul 18 01:54:31.242211 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:31.242143 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pvq8t" Jul 18 01:54:31.252367 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:31.252349 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b8nbn" Jul 18 01:54:31.424297 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:31.424267 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-pvq8t"] Jul 18 01:54:31.427463 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:54:31.427442 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc0e8558a_a0ba_4d9c_a4bc_4cab55db2a77.slice/crio-c567b34ceb7a0899bf87ab366b98cd351dc621de12325f0258290b8da91e4fd8 WatchSource:0}: Error finding container c567b34ceb7a0899bf87ab366b98cd351dc621de12325f0258290b8da91e4fd8: Status 404 returned error can't find the container with id c567b34ceb7a0899bf87ab366b98cd351dc621de12325f0258290b8da91e4fd8 Jul 18 01:54:31.442565 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:31.442535 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-b8nbn"] Jul 18 01:54:31.443803 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:54:31.443760 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a20675e_03ef_44a5_8663_ead09b5e6774.slice/crio-1f111314a84a6da8a22c58f5c052f946e24beabfe9c3d56b403c8d9d12d82c44 WatchSource:0}: Error finding container 1f111314a84a6da8a22c58f5c052f946e24beabfe9c3d56b403c8d9d12d82c44: Status 404 returned error can't find the container with id 1f111314a84a6da8a22c58f5c052f946e24beabfe9c3d56b403c8d9d12d82c44 Jul 18 01:54:32.313229 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:32.313191 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b8nbn" event={"ID":"8a20675e-03ef-44a5-8663-ead09b5e6774","Type":"ContainerStarted","Data":"1f111314a84a6da8a22c58f5c052f946e24beabfe9c3d56b403c8d9d12d82c44"} Jul 18 01:54:32.314837 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:32.314806 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pvq8t" event={"ID":"c0e8558a-a0ba-4d9c-a4bc-4cab55db2a77","Type":"ContainerStarted","Data":"c567b34ceb7a0899bf87ab366b98cd351dc621de12325f0258290b8da91e4fd8"} Jul 18 01:54:33.708986 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:33.708957 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc"] Jul 18 01:54:33.709429 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:33.709165 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" podUID="76e66dae-0b7d-4e97-9341-0eebd53114f8" containerName="discovery" containerID="cri-o://5ddea3c3d21a62107354e01c3e372d37d9867b8b5f2b757755fe421ce5de0771" gracePeriod=30 Jul 18 01:54:34.303021 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.302996 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:34.329426 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.329401 2548 generic.go:356] "Generic (PLEG): container finished" podID="76e66dae-0b7d-4e97-9341-0eebd53114f8" containerID="5ddea3c3d21a62107354e01c3e372d37d9867b8b5f2b757755fe421ce5de0771" exitCode=0 Jul 18 01:54:34.329556 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.329441 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" Jul 18 01:54:34.329556 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.329478 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" event={"ID":"76e66dae-0b7d-4e97-9341-0eebd53114f8","Type":"ContainerDied","Data":"5ddea3c3d21a62107354e01c3e372d37d9867b8b5f2b757755fe421ce5de0771"} Jul 18 01:54:34.329556 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.329523 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc" event={"ID":"76e66dae-0b7d-4e97-9341-0eebd53114f8","Type":"ContainerDied","Data":"15a13d2903cf7a4b9dd0efd1fb7a7e5d4644188d5d603a47a51dd319ebdd761b"} Jul 18 01:54:34.329556 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.329550 2548 scope.go:117] "RemoveContainer" containerID="5ddea3c3d21a62107354e01c3e372d37d9867b8b5f2b757755fe421ce5de0771" Jul 18 01:54:34.331638 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.331614 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b8nbn" event={"ID":"8a20675e-03ef-44a5-8663-ead09b5e6774","Type":"ContainerStarted","Data":"1d2a9b9d442afe5ec97fabd914009abcaa753e0cd1e9762c74f71b31e9b6adf4"} Jul 18 01:54:34.331739 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.331672 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b8nbn" Jul 18 01:54:34.333216 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.333194 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pvq8t" event={"ID":"c0e8558a-a0ba-4d9c-a4bc-4cab55db2a77","Type":"ContainerStarted","Data":"eb4a39f44fb003d85c4903d57ddb1d633b29c6e0fc4d15be6ff30e0e24fc156d"} Jul 18 01:54:34.333294 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.333244 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pvq8t" Jul 18 01:54:34.338472 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.338456 2548 scope.go:117] "RemoveContainer" containerID="5ddea3c3d21a62107354e01c3e372d37d9867b8b5f2b757755fe421ce5de0771" Jul 18 01:54:34.338756 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:54:34.338725 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ddea3c3d21a62107354e01c3e372d37d9867b8b5f2b757755fe421ce5de0771\": container with ID starting with 5ddea3c3d21a62107354e01c3e372d37d9867b8b5f2b757755fe421ce5de0771 not found: ID does not exist" containerID="5ddea3c3d21a62107354e01c3e372d37d9867b8b5f2b757755fe421ce5de0771" Jul 18 01:54:34.338841 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.338792 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ddea3c3d21a62107354e01c3e372d37d9867b8b5f2b757755fe421ce5de0771"} err="failed to get container status \"5ddea3c3d21a62107354e01c3e372d37d9867b8b5f2b757755fe421ce5de0771\": rpc error: code = NotFound desc = could not find container \"5ddea3c3d21a62107354e01c3e372d37d9867b8b5f2b757755fe421ce5de0771\": container with ID starting with 5ddea3c3d21a62107354e01c3e372d37d9867b8b5f2b757755fe421ce5de0771 not found: ID does not exist" Jul 18 01:54:34.357110 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.357089 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/76e66dae-0b7d-4e97-9341-0eebd53114f8-local-certs\") pod \"76e66dae-0b7d-4e97-9341-0eebd53114f8\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " Jul 18 01:54:34.357208 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.357120 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-csr-ca-configmap\") pod \"76e66dae-0b7d-4e97-9341-0eebd53114f8\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " Jul 18 01:54:34.357208 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.357165 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-token\") pod \"76e66dae-0b7d-4e97-9341-0eebd53114f8\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " Jul 18 01:54:34.357208 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.357190 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-csr-dns-cert\") pod \"76e66dae-0b7d-4e97-9341-0eebd53114f8\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " Jul 18 01:54:34.357353 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.357213 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-kubeconfig\") pod \"76e66dae-0b7d-4e97-9341-0eebd53114f8\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " Jul 18 01:54:34.357353 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.357235 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-cacerts\") pod \"76e66dae-0b7d-4e97-9341-0eebd53114f8\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " Jul 18 01:54:34.357353 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.357300 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzn97\" (UniqueName: \"kubernetes.io/projected/76e66dae-0b7d-4e97-9341-0eebd53114f8-kube-api-access-xzn97\") pod \"76e66dae-0b7d-4e97-9341-0eebd53114f8\" (UID: \"76e66dae-0b7d-4e97-9341-0eebd53114f8\") " Jul 18 01:54:34.357685 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.357643 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "76e66dae-0b7d-4e97-9341-0eebd53114f8" (UID: "76e66dae-0b7d-4e97-9341-0eebd53114f8"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:54:34.359814 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.359773 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76e66dae-0b7d-4e97-9341-0eebd53114f8-local-certs" (OuterVolumeSpecName: "local-certs") pod "76e66dae-0b7d-4e97-9341-0eebd53114f8" (UID: "76e66dae-0b7d-4e97-9341-0eebd53114f8"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:54:34.360161 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.360134 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "76e66dae-0b7d-4e97-9341-0eebd53114f8" (UID: "76e66dae-0b7d-4e97-9341-0eebd53114f8"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:54:34.360161 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.360151 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-cacerts" (OuterVolumeSpecName: "cacerts") pod "76e66dae-0b7d-4e97-9341-0eebd53114f8" (UID: "76e66dae-0b7d-4e97-9341-0eebd53114f8"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:54:34.360316 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.360165 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-token" (OuterVolumeSpecName: "istio-token") pod "76e66dae-0b7d-4e97-9341-0eebd53114f8" (UID: "76e66dae-0b7d-4e97-9341-0eebd53114f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:54:34.360316 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.360297 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76e66dae-0b7d-4e97-9341-0eebd53114f8-kube-api-access-xzn97" (OuterVolumeSpecName: "kube-api-access-xzn97") pod "76e66dae-0b7d-4e97-9341-0eebd53114f8" (UID: "76e66dae-0b7d-4e97-9341-0eebd53114f8"). InnerVolumeSpecName "kube-api-access-xzn97". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:54:34.360395 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.360338 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "76e66dae-0b7d-4e97-9341-0eebd53114f8" (UID: "76e66dae-0b7d-4e97-9341-0eebd53114f8"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 01:54:34.387336 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.387299 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b8nbn" podStartSLOduration=1.6605584260000001 podStartE2EDuration="4.387285405s" totalImagesPullingTime="2.726726979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:34.387050704 +0000 UTC m=+383.239794049" watchObservedRunningTime="2026-07-18 01:54:34.387285405 +0000 UTC m=+383.240028748" Jul 18 01:54:34.421189 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.421150 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pvq8t" podStartSLOduration=1.673230065 podStartE2EDuration="4.421135578s" totalImagesPullingTime="2.747905513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:34.41966455 +0000 UTC m=+383.272407893" watchObservedRunningTime="2026-07-18 01:54:34.421135578 +0000 UTC m=+383.273878922" Jul 18 01:54:34.458189 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.458169 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzn97\" (UniqueName: \"kubernetes.io/projected/76e66dae-0b7d-4e97-9341-0eebd53114f8-kube-api-access-xzn97\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:54:34.458189 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.458189 2548 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/76e66dae-0b7d-4e97-9341-0eebd53114f8-local-certs\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:54:34.458339 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.458282 2548 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-csr-ca-configmap\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:54:34.458339 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.458301 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-token\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:54:34.463361 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.458633 2548 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-csr-dns-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:54:34.463361 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.458683 2548 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-istio-kubeconfig\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:54:34.463361 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.458701 2548 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/76e66dae-0b7d-4e97-9341-0eebd53114f8-cacerts\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:54:34.651944 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.651914 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc"] Jul 18 01:54:34.661222 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:34.661202 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5rgbc"] Jul 18 01:54:35.549029 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:35.548994 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76e66dae-0b7d-4e97-9341-0eebd53114f8" path="/var/lib/kubelet/pods/76e66dae-0b7d-4e97-9341-0eebd53114f8/volumes" Jul 18 01:54:37.288389 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:37.288356 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-g57qp" Jul 18 01:54:37.290647 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:54:37.290624 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-bk8lx" Jul 18 01:55:05.341385 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:05.341356 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-pvq8t" Jul 18 01:55:05.344490 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:05.344469 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-b8nbn" Jul 18 01:55:20.225798 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.225752 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pzshs"] Jul 18 01:55:20.226158 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.226076 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76e66dae-0b7d-4e97-9341-0eebd53114f8" containerName="discovery" Jul 18 01:55:20.226158 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.226087 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="76e66dae-0b7d-4e97-9341-0eebd53114f8" containerName="discovery" Jul 18 01:55:20.226158 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.226148 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="76e66dae-0b7d-4e97-9341-0eebd53114f8" containerName="discovery" Jul 18 01:55:20.232388 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.232368 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pzshs" Jul 18 01:55:20.234926 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.234899 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 18 01:55:20.235293 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.235276 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 18 01:55:20.235920 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.235902 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-sxm87\"" Jul 18 01:55:20.245103 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.245081 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pzshs"] Jul 18 01:55:20.295324 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.295298 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4gdq\" (UniqueName: \"kubernetes.io/projected/c3d81b3b-c054-40d0-9ad0-c34a4261bde8-kube-api-access-p4gdq\") pod \"kuadrant-operator-catalog-pzshs\" (UID: \"c3d81b3b-c054-40d0-9ad0-c34a4261bde8\") " pod="kuadrant-system/kuadrant-operator-catalog-pzshs" Jul 18 01:55:20.396195 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.396174 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4gdq\" (UniqueName: \"kubernetes.io/projected/c3d81b3b-c054-40d0-9ad0-c34a4261bde8-kube-api-access-p4gdq\") pod \"kuadrant-operator-catalog-pzshs\" (UID: \"c3d81b3b-c054-40d0-9ad0-c34a4261bde8\") " pod="kuadrant-system/kuadrant-operator-catalog-pzshs" Jul 18 01:55:20.404179 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.404157 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4gdq\" (UniqueName: \"kubernetes.io/projected/c3d81b3b-c054-40d0-9ad0-c34a4261bde8-kube-api-access-p4gdq\") pod \"kuadrant-operator-catalog-pzshs\" (UID: \"c3d81b3b-c054-40d0-9ad0-c34a4261bde8\") " pod="kuadrant-system/kuadrant-operator-catalog-pzshs" Jul 18 01:55:20.542422 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.542367 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pzshs" Jul 18 01:55:20.582033 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.582008 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pzshs"] Jul 18 01:55:20.672339 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.672265 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pzshs"] Jul 18 01:55:20.674915 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:55:20.674891 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d81b3b_c054_40d0_9ad0_c34a4261bde8.slice/crio-b33125f77284b405b4c3ec20ba8d40c75cf3ab2085032670c824e993343c7776 WatchSource:0}: Error finding container b33125f77284b405b4c3ec20ba8d40c75cf3ab2085032670c824e993343c7776: Status 404 returned error can't find the container with id b33125f77284b405b4c3ec20ba8d40c75cf3ab2085032670c824e993343c7776 Jul 18 01:55:20.790710 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.790687 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k4dhq"] Jul 18 01:55:20.794933 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.794892 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k4dhq" Jul 18 01:55:20.804111 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.804081 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k4dhq"] Jul 18 01:55:20.899593 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:20.899563 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5rnw\" (UniqueName: \"kubernetes.io/projected/3045c3c3-9924-4079-a61e-37accc8c1e6d-kube-api-access-b5rnw\") pod \"kuadrant-operator-catalog-k4dhq\" (UID: \"3045c3c3-9924-4079-a61e-37accc8c1e6d\") " pod="kuadrant-system/kuadrant-operator-catalog-k4dhq" Jul 18 01:55:21.000120 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:21.000093 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5rnw\" (UniqueName: \"kubernetes.io/projected/3045c3c3-9924-4079-a61e-37accc8c1e6d-kube-api-access-b5rnw\") pod \"kuadrant-operator-catalog-k4dhq\" (UID: \"3045c3c3-9924-4079-a61e-37accc8c1e6d\") " pod="kuadrant-system/kuadrant-operator-catalog-k4dhq" Jul 18 01:55:21.008838 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:21.008815 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5rnw\" (UniqueName: \"kubernetes.io/projected/3045c3c3-9924-4079-a61e-37accc8c1e6d-kube-api-access-b5rnw\") pod \"kuadrant-operator-catalog-k4dhq\" (UID: \"3045c3c3-9924-4079-a61e-37accc8c1e6d\") " pod="kuadrant-system/kuadrant-operator-catalog-k4dhq" Jul 18 01:55:21.110318 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:21.110244 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-k4dhq" Jul 18 01:55:21.271089 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:21.271065 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-k4dhq"] Jul 18 01:55:21.276647 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:55:21.276620 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3045c3c3_9924_4079_a61e_37accc8c1e6d.slice/crio-49b133ea44af88a4326c04a11447639af98659fd8fafec33c41bc811f4085f5f WatchSource:0}: Error finding container 49b133ea44af88a4326c04a11447639af98659fd8fafec33c41bc811f4085f5f: Status 404 returned error can't find the container with id 49b133ea44af88a4326c04a11447639af98659fd8fafec33c41bc811f4085f5f Jul 18 01:55:21.539336 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:21.539302 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k4dhq" event={"ID":"3045c3c3-9924-4079-a61e-37accc8c1e6d","Type":"ContainerStarted","Data":"49b133ea44af88a4326c04a11447639af98659fd8fafec33c41bc811f4085f5f"} Jul 18 01:55:21.540244 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:21.540222 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pzshs" event={"ID":"c3d81b3b-c054-40d0-9ad0-c34a4261bde8","Type":"ContainerStarted","Data":"b33125f77284b405b4c3ec20ba8d40c75cf3ab2085032670c824e993343c7776"} Jul 18 01:55:23.551250 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:23.551214 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-k4dhq" event={"ID":"3045c3c3-9924-4079-a61e-37accc8c1e6d","Type":"ContainerStarted","Data":"1bc4e1d8540ffd4e953fe99121c0e9196bc4dd80fd8c71311e861a4b00280a25"} Jul 18 01:55:23.552597 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:23.552561 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pzshs" event={"ID":"c3d81b3b-c054-40d0-9ad0-c34a4261bde8","Type":"ContainerStarted","Data":"cabae168f2970f5d1a5ad83c06f56731580334464a3f0d398c778334f9274064"} Jul 18 01:55:23.552718 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:23.552633 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-pzshs" podUID="c3d81b3b-c054-40d0-9ad0-c34a4261bde8" containerName="registry-server" containerID="cri-o://cabae168f2970f5d1a5ad83c06f56731580334464a3f0d398c778334f9274064" gracePeriod=2 Jul 18 01:55:23.569460 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:23.569418 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-k4dhq" podStartSLOduration=2.163839039 podStartE2EDuration="3.569402522s" totalImagesPullingTime="1.405563483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:23.568620198 +0000 UTC m=+432.421363543" watchObservedRunningTime="2026-07-18 01:55:23.569402522 +0000 UTC m=+432.422145865" Jul 18 01:55:23.585128 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:23.585083 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-pzshs" podStartSLOduration=1.581316479 podStartE2EDuration="3.585068549s" totalImagesPullingTime="2.00375207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:23.584086011 +0000 UTC m=+432.436829356" watchObservedRunningTime="2026-07-18 01:55:23.585068549 +0000 UTC m=+432.437811943" Jul 18 01:55:23.805805 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:23.805743 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pzshs" Jul 18 01:55:23.822438 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:23.822415 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4gdq\" (UniqueName: \"kubernetes.io/projected/c3d81b3b-c054-40d0-9ad0-c34a4261bde8-kube-api-access-p4gdq\") pod \"c3d81b3b-c054-40d0-9ad0-c34a4261bde8\" (UID: \"c3d81b3b-c054-40d0-9ad0-c34a4261bde8\") " Jul 18 01:55:23.824437 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:23.824415 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3d81b3b-c054-40d0-9ad0-c34a4261bde8-kube-api-access-p4gdq" (OuterVolumeSpecName: "kube-api-access-p4gdq") pod "c3d81b3b-c054-40d0-9ad0-c34a4261bde8" (UID: "c3d81b3b-c054-40d0-9ad0-c34a4261bde8"). InnerVolumeSpecName "kube-api-access-p4gdq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:55:23.923071 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:23.923039 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4gdq\" (UniqueName: \"kubernetes.io/projected/c3d81b3b-c054-40d0-9ad0-c34a4261bde8-kube-api-access-p4gdq\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:55:24.557448 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:24.557416 2548 generic.go:356] "Generic (PLEG): container finished" podID="c3d81b3b-c054-40d0-9ad0-c34a4261bde8" containerID="cabae168f2970f5d1a5ad83c06f56731580334464a3f0d398c778334f9274064" exitCode=0 Jul 18 01:55:24.557448 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:24.557454 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pzshs" Jul 18 01:55:24.557970 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:24.557498 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pzshs" event={"ID":"c3d81b3b-c054-40d0-9ad0-c34a4261bde8","Type":"ContainerDied","Data":"cabae168f2970f5d1a5ad83c06f56731580334464a3f0d398c778334f9274064"} Jul 18 01:55:24.557970 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:24.557545 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pzshs" event={"ID":"c3d81b3b-c054-40d0-9ad0-c34a4261bde8","Type":"ContainerDied","Data":"b33125f77284b405b4c3ec20ba8d40c75cf3ab2085032670c824e993343c7776"} Jul 18 01:55:24.557970 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:24.557568 2548 scope.go:117] "RemoveContainer" containerID="cabae168f2970f5d1a5ad83c06f56731580334464a3f0d398c778334f9274064" Jul 18 01:55:24.566992 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:24.566976 2548 scope.go:117] "RemoveContainer" containerID="cabae168f2970f5d1a5ad83c06f56731580334464a3f0d398c778334f9274064" Jul 18 01:55:24.567244 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:55:24.567225 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cabae168f2970f5d1a5ad83c06f56731580334464a3f0d398c778334f9274064\": container with ID starting with cabae168f2970f5d1a5ad83c06f56731580334464a3f0d398c778334f9274064 not found: ID does not exist" containerID="cabae168f2970f5d1a5ad83c06f56731580334464a3f0d398c778334f9274064" Jul 18 01:55:24.567284 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:24.567254 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cabae168f2970f5d1a5ad83c06f56731580334464a3f0d398c778334f9274064"} err="failed to get container status \"cabae168f2970f5d1a5ad83c06f56731580334464a3f0d398c778334f9274064\": rpc error: code = NotFound desc = could not find container \"cabae168f2970f5d1a5ad83c06f56731580334464a3f0d398c778334f9274064\": container with ID starting with cabae168f2970f5d1a5ad83c06f56731580334464a3f0d398c778334f9274064 not found: ID does not exist" Jul 18 01:55:24.583028 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:24.580349 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pzshs"] Jul 18 01:55:24.584987 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:24.584961 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pzshs"] Jul 18 01:55:25.548653 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:25.548618 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3d81b3b-c054-40d0-9ad0-c34a4261bde8" path="/var/lib/kubelet/pods/c3d81b3b-c054-40d0-9ad0-c34a4261bde8/volumes" Jul 18 01:55:31.110517 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:31.110477 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-k4dhq" Jul 18 01:55:31.110980 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:31.110546 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-k4dhq" Jul 18 01:55:31.133162 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:31.133136 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-k4dhq" Jul 18 01:55:31.612504 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:31.612478 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-k4dhq" Jul 18 01:55:52.454546 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:52.454507 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-zz8tb"] Jul 18 01:55:52.454935 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:52.454860 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3d81b3b-c054-40d0-9ad0-c34a4261bde8" containerName="registry-server" Jul 18 01:55:52.454935 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:52.454872 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3d81b3b-c054-40d0-9ad0-c34a4261bde8" containerName="registry-server" Jul 18 01:55:52.455003 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:52.454948 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3d81b3b-c054-40d0-9ad0-c34a4261bde8" containerName="registry-server" Jul 18 01:55:52.457812 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:52.457797 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zz8tb" Jul 18 01:55:52.460420 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:52.460400 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qdhsm\"" Jul 18 01:55:52.460610 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:52.460597 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 18 01:55:52.470155 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:52.470132 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-zz8tb"] Jul 18 01:55:52.547546 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:52.547504 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd72r\" (UniqueName: \"kubernetes.io/projected/7a028ff6-cd86-4e61-bca8-aee5c7d8cdb8-kube-api-access-dd72r\") pod \"dns-operator-controller-manager-864dd78c4c-zz8tb\" (UID: \"7a028ff6-cd86-4e61-bca8-aee5c7d8cdb8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zz8tb" Jul 18 01:55:52.648100 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:52.648063 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd72r\" (UniqueName: \"kubernetes.io/projected/7a028ff6-cd86-4e61-bca8-aee5c7d8cdb8-kube-api-access-dd72r\") pod \"dns-operator-controller-manager-864dd78c4c-zz8tb\" (UID: \"7a028ff6-cd86-4e61-bca8-aee5c7d8cdb8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zz8tb" Jul 18 01:55:52.665578 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:52.665549 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd72r\" (UniqueName: \"kubernetes.io/projected/7a028ff6-cd86-4e61-bca8-aee5c7d8cdb8-kube-api-access-dd72r\") pod \"dns-operator-controller-manager-864dd78c4c-zz8tb\" (UID: \"7a028ff6-cd86-4e61-bca8-aee5c7d8cdb8\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zz8tb" Jul 18 01:55:52.767412 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:52.767343 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zz8tb" Jul 18 01:55:52.913090 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:52.913067 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-zz8tb"] Jul 18 01:55:52.915938 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:55:52.915908 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a028ff6_cd86_4e61_bca8_aee5c7d8cdb8.slice/crio-11173ed72e09874089e86048df9ea8d16597d558668aea1cb723d76b54af4b86 WatchSource:0}: Error finding container 11173ed72e09874089e86048df9ea8d16597d558668aea1cb723d76b54af4b86: Status 404 returned error can't find the container with id 11173ed72e09874089e86048df9ea8d16597d558668aea1cb723d76b54af4b86 Jul 18 01:55:53.681425 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:53.681386 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zz8tb" event={"ID":"7a028ff6-cd86-4e61-bca8-aee5c7d8cdb8","Type":"ContainerStarted","Data":"11173ed72e09874089e86048df9ea8d16597d558668aea1cb723d76b54af4b86"} Jul 18 01:55:54.687375 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:54.687336 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zz8tb" event={"ID":"7a028ff6-cd86-4e61-bca8-aee5c7d8cdb8","Type":"ContainerStarted","Data":"64c41b4277219890c3c7262f714e4e0f6a2ec10e1d4a840eb55373df5ef813eb"} Jul 18 01:55:54.687751 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:54.687464 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zz8tb" Jul 18 01:55:54.713081 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:54.712985 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zz8tb" podStartSLOduration=1.181082311 podStartE2EDuration="2.712968041s" totalImagesPullingTime="1.53188573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:54.710406542 +0000 UTC m=+463.563149914" watchObservedRunningTime="2026-07-18 01:55:54.712968041 +0000 UTC m=+463.565711384" Jul 18 01:55:58.033942 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:58.033903 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z"] Jul 18 01:55:58.037368 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:58.037348 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" Jul 18 01:55:58.040282 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:58.040264 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qsppv\"" Jul 18 01:55:58.049653 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:58.049603 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z"] Jul 18 01:55:58.091595 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:58.091570 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8tfl\" (UniqueName: \"kubernetes.io/projected/af130afb-9312-478a-a556-5cc2af4a8d99-kube-api-access-s8tfl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r7z7z\" (UID: \"af130afb-9312-478a-a556-5cc2af4a8d99\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" Jul 18 01:55:58.091722 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:58.091657 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af130afb-9312-478a-a556-5cc2af4a8d99-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r7z7z\" (UID: \"af130afb-9312-478a-a556-5cc2af4a8d99\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" Jul 18 01:55:58.192166 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:58.192129 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8tfl\" (UniqueName: \"kubernetes.io/projected/af130afb-9312-478a-a556-5cc2af4a8d99-kube-api-access-s8tfl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r7z7z\" (UID: \"af130afb-9312-478a-a556-5cc2af4a8d99\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" Jul 18 01:55:58.192323 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:58.192213 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af130afb-9312-478a-a556-5cc2af4a8d99-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r7z7z\" (UID: \"af130afb-9312-478a-a556-5cc2af4a8d99\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" Jul 18 01:55:58.192535 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:58.192516 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af130afb-9312-478a-a556-5cc2af4a8d99-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r7z7z\" (UID: \"af130afb-9312-478a-a556-5cc2af4a8d99\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" Jul 18 01:55:58.201654 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:58.201623 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8tfl\" (UniqueName: \"kubernetes.io/projected/af130afb-9312-478a-a556-5cc2af4a8d99-kube-api-access-s8tfl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r7z7z\" (UID: \"af130afb-9312-478a-a556-5cc2af4a8d99\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" Jul 18 01:55:58.347059 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:58.346965 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" Jul 18 01:55:58.481441 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:58.481408 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z"] Jul 18 01:55:58.485029 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:55:58.485004 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf130afb_9312_478a_a556_5cc2af4a8d99.slice/crio-b2a4ade9d09384341c076d96d7b34eff8d3370e70288f55fa4407573c95d87a0 WatchSource:0}: Error finding container b2a4ade9d09384341c076d96d7b34eff8d3370e70288f55fa4407573c95d87a0: Status 404 returned error can't find the container with id b2a4ade9d09384341c076d96d7b34eff8d3370e70288f55fa4407573c95d87a0 Jul 18 01:55:58.706927 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:55:58.706894 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" event={"ID":"af130afb-9312-478a-a556-5cc2af4a8d99","Type":"ContainerStarted","Data":"b2a4ade9d09384341c076d96d7b34eff8d3370e70288f55fa4407573c95d87a0"} Jul 18 01:56:01.070165 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:01.070129 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z"] Jul 18 01:56:01.074148 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:01.074127 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z" Jul 18 01:56:01.077628 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:01.077607 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-6s6s7\"" Jul 18 01:56:01.104435 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:01.104407 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z"] Jul 18 01:56:01.218402 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:01.218367 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf5pw\" (UniqueName: \"kubernetes.io/projected/ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd-kube-api-access-lf5pw\") pod \"limitador-operator-controller-manager-859d7b78bd-wz68z\" (UID: \"ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z" Jul 18 01:56:01.318945 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:01.318904 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf5pw\" (UniqueName: \"kubernetes.io/projected/ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd-kube-api-access-lf5pw\") pod \"limitador-operator-controller-manager-859d7b78bd-wz68z\" (UID: \"ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z" Jul 18 01:56:01.331911 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:01.331833 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf5pw\" (UniqueName: \"kubernetes.io/projected/ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd-kube-api-access-lf5pw\") pod \"limitador-operator-controller-manager-859d7b78bd-wz68z\" (UID: \"ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z" Jul 18 01:56:01.388294 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:01.388258 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z" Jul 18 01:56:02.294135 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:02.294113 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z"] Jul 18 01:56:02.296326 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:56:02.296303 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca74ce5f_b28e_4f09_a63a_c9a88ab1a3dd.slice/crio-addd0bf1feac737f26a236d09d2ffcf583c9def735009bcd7df89aa499001534 WatchSource:0}: Error finding container addd0bf1feac737f26a236d09d2ffcf583c9def735009bcd7df89aa499001534: Status 404 returned error can't find the container with id addd0bf1feac737f26a236d09d2ffcf583c9def735009bcd7df89aa499001534 Jul 18 01:56:02.729459 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:02.729422 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" event={"ID":"af130afb-9312-478a-a556-5cc2af4a8d99","Type":"ContainerStarted","Data":"600abf58f867fbb1f69e7c756f3daa179bdeba93d656872f233998d4c2881aa4"} Jul 18 01:56:02.729619 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:02.729540 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" Jul 18 01:56:02.730875 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:02.730854 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z" event={"ID":"ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd","Type":"ContainerStarted","Data":"addd0bf1feac737f26a236d09d2ffcf583c9def735009bcd7df89aa499001534"} Jul 18 01:56:02.766004 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:02.765956 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" podStartSLOduration=1.035187053 podStartE2EDuration="4.76593873s" totalImagesPullingTime="3.730751677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:02.764473412 +0000 UTC m=+471.617216760" watchObservedRunningTime="2026-07-18 01:56:02.76593873 +0000 UTC m=+471.618682075" Jul 18 01:56:04.292648 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:04.292620 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-x7bs7"] Jul 18 01:56:04.296061 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:04.296042 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x7bs7" Jul 18 01:56:04.298388 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:04.298373 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-vlckk\"" Jul 18 01:56:04.308371 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:04.308349 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-x7bs7"] Jul 18 01:56:04.445675 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:04.445648 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2fgx\" (UniqueName: \"kubernetes.io/projected/e0ab8e91-24ee-4c70-bf9b-e6707293fb03-kube-api-access-s2fgx\") pod \"authorino-operator-75cc8d5cb7-x7bs7\" (UID: \"e0ab8e91-24ee-4c70-bf9b-e6707293fb03\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-x7bs7" Jul 18 01:56:04.546128 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:04.546060 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2fgx\" (UniqueName: \"kubernetes.io/projected/e0ab8e91-24ee-4c70-bf9b-e6707293fb03-kube-api-access-s2fgx\") pod \"authorino-operator-75cc8d5cb7-x7bs7\" (UID: \"e0ab8e91-24ee-4c70-bf9b-e6707293fb03\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-x7bs7" Jul 18 01:56:04.554888 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:04.554867 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2fgx\" (UniqueName: \"kubernetes.io/projected/e0ab8e91-24ee-4c70-bf9b-e6707293fb03-kube-api-access-s2fgx\") pod \"authorino-operator-75cc8d5cb7-x7bs7\" (UID: \"e0ab8e91-24ee-4c70-bf9b-e6707293fb03\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-x7bs7" Jul 18 01:56:04.606204 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:04.606176 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x7bs7" Jul 18 01:56:04.739066 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:04.739032 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-x7bs7"] Jul 18 01:56:04.741296 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:04.741269 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z" event={"ID":"ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd","Type":"ContainerStarted","Data":"5a29105182effb3e85bc788425dc97396e12aa3a96b1ddfaecde97aafb8de44c"} Jul 18 01:56:04.741569 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:04.741553 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z" Jul 18 01:56:04.742325 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:56:04.742307 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0ab8e91_24ee_4c70_bf9b_e6707293fb03.slice/crio-eb119978ebb67168694e06568d0d48809ca017e9a13d5a281e66ec459c579a41 WatchSource:0}: Error finding container eb119978ebb67168694e06568d0d48809ca017e9a13d5a281e66ec459c579a41: Status 404 returned error can't find the container with id eb119978ebb67168694e06568d0d48809ca017e9a13d5a281e66ec459c579a41 Jul 18 01:56:04.762125 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:04.762079 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z" podStartSLOduration=2.322210539 podStartE2EDuration="3.762064076s" totalImagesPullingTime="1.439853537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:04.760769816 +0000 UTC m=+473.613513160" watchObservedRunningTime="2026-07-18 01:56:04.762064076 +0000 UTC m=+473.614807420" Jul 18 01:56:05.205133 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.205096 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9nfft"] Jul 18 01:56:05.208829 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.208799 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9nfft" Jul 18 01:56:05.211172 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.211148 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 18 01:56:05.211472 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.211450 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 18 01:56:05.211566 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.211467 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qrwf8\"" Jul 18 01:56:05.216968 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.216941 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9nfft"] Jul 18 01:56:05.353463 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.353431 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/806dc654-0543-4b3b-baec-68d01a5db966-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9nfft\" (UID: \"806dc654-0543-4b3b-baec-68d01a5db966\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9nfft" Jul 18 01:56:05.353920 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.353509 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/806dc654-0543-4b3b-baec-68d01a5db966-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9nfft\" (UID: \"806dc654-0543-4b3b-baec-68d01a5db966\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9nfft" Jul 18 01:56:05.353920 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.353547 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjjpk\" (UniqueName: \"kubernetes.io/projected/806dc654-0543-4b3b-baec-68d01a5db966-kube-api-access-kjjpk\") pod \"kuadrant-console-plugin-66b5774985-9nfft\" (UID: \"806dc654-0543-4b3b-baec-68d01a5db966\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9nfft" Jul 18 01:56:05.454862 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.454830 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/806dc654-0543-4b3b-baec-68d01a5db966-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9nfft\" (UID: \"806dc654-0543-4b3b-baec-68d01a5db966\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9nfft" Jul 18 01:56:05.455016 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.454884 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjjpk\" (UniqueName: \"kubernetes.io/projected/806dc654-0543-4b3b-baec-68d01a5db966-kube-api-access-kjjpk\") pod \"kuadrant-console-plugin-66b5774985-9nfft\" (UID: \"806dc654-0543-4b3b-baec-68d01a5db966\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9nfft" Jul 18 01:56:05.455016 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.454957 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/806dc654-0543-4b3b-baec-68d01a5db966-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9nfft\" (UID: \"806dc654-0543-4b3b-baec-68d01a5db966\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9nfft" Jul 18 01:56:05.455760 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.455701 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/806dc654-0543-4b3b-baec-68d01a5db966-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-9nfft\" (UID: \"806dc654-0543-4b3b-baec-68d01a5db966\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9nfft" Jul 18 01:56:05.458174 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.458152 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/806dc654-0543-4b3b-baec-68d01a5db966-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-9nfft\" (UID: \"806dc654-0543-4b3b-baec-68d01a5db966\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9nfft" Jul 18 01:56:05.464772 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.464747 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjjpk\" (UniqueName: \"kubernetes.io/projected/806dc654-0543-4b3b-baec-68d01a5db966-kube-api-access-kjjpk\") pod \"kuadrant-console-plugin-66b5774985-9nfft\" (UID: \"806dc654-0543-4b3b-baec-68d01a5db966\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9nfft" Jul 18 01:56:05.520566 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.520520 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9nfft" Jul 18 01:56:05.695507 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.695471 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zz8tb" Jul 18 01:56:05.705952 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.704758 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-9nfft"] Jul 18 01:56:05.707990 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:56:05.707964 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod806dc654_0543_4b3b_baec_68d01a5db966.slice/crio-97a33a8e474dcd48fee1dbb150fe12f80e383ebb6c0c9970e1b43bafc77125c1 WatchSource:0}: Error finding container 97a33a8e474dcd48fee1dbb150fe12f80e383ebb6c0c9970e1b43bafc77125c1: Status 404 returned error can't find the container with id 97a33a8e474dcd48fee1dbb150fe12f80e383ebb6c0c9970e1b43bafc77125c1 Jul 18 01:56:05.748947 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.748915 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x7bs7" event={"ID":"e0ab8e91-24ee-4c70-bf9b-e6707293fb03","Type":"ContainerStarted","Data":"eb119978ebb67168694e06568d0d48809ca017e9a13d5a281e66ec459c579a41"} Jul 18 01:56:05.750926 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:05.750875 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9nfft" event={"ID":"806dc654-0543-4b3b-baec-68d01a5db966","Type":"ContainerStarted","Data":"97a33a8e474dcd48fee1dbb150fe12f80e383ebb6c0c9970e1b43bafc77125c1"} Jul 18 01:56:07.764357 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:07.764322 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x7bs7" event={"ID":"e0ab8e91-24ee-4c70-bf9b-e6707293fb03","Type":"ContainerStarted","Data":"bdc13b6da0d8e94615118c271465625e225bae9bf4f7e9bc0f82e78520fb68a1"} Jul 18 01:56:07.764853 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:07.764519 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x7bs7" Jul 18 01:56:07.782860 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:07.782813 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x7bs7" podStartSLOduration=1.6748071960000002 podStartE2EDuration="3.782794698s" totalImagesPullingTime="2.107987502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:07.781878153 +0000 UTC m=+476.634621508" watchObservedRunningTime="2026-07-18 01:56:07.782794698 +0000 UTC m=+476.635538042" Jul 18 01:56:13.738341 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:13.738309 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" Jul 18 01:56:15.410799 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.410678 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p"] Jul 18 01:56:15.438947 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.438911 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p"] Jul 18 01:56:15.439120 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.439060 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p" Jul 18 01:56:15.467514 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.467483 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z"] Jul 18 01:56:15.467701 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.467673 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" podUID="af130afb-9312-478a-a556-5cc2af4a8d99" containerName="manager" containerID="cri-o://600abf58f867fbb1f69e7c756f3daa179bdeba93d656872f233998d4c2881aa4" gracePeriod=2 Jul 18 01:56:15.474285 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.474262 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z"] Jul 18 01:56:15.487593 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.486809 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p"] Jul 18 01:56:15.487593 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:56:15.487451 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-696mp], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p" podUID="b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5" Jul 18 01:56:15.500446 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.498018 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr"] Jul 18 01:56:15.500446 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.498523 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af130afb-9312-478a-a556-5cc2af4a8d99" containerName="manager" Jul 18 01:56:15.500446 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.498539 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="af130afb-9312-478a-a556-5cc2af4a8d99" containerName="manager" Jul 18 01:56:15.500795 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.500756 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="af130afb-9312-478a-a556-5cc2af4a8d99" containerName="manager" Jul 18 01:56:15.505445 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.505422 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p"] Jul 18 01:56:15.505562 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.505541 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" Jul 18 01:56:15.512424 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.512398 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78"] Jul 18 01:56:15.518294 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.518270 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z"] Jul 18 01:56:15.518691 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.518658 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z" podUID="ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd" containerName="manager" containerID="cri-o://5a29105182effb3e85bc788425dc97396e12aa3a96b1ddfaecde97aafb8de44c" gracePeriod=2 Jul 18 01:56:15.519068 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.519045 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" Jul 18 01:56:15.520196 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.520178 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr"] Jul 18 01:56:15.521360 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.521340 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z" Jul 18 01:56:15.533807 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.533699 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z"] Jul 18 01:56:15.536182 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.536146 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78"] Jul 18 01:56:15.540262 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:56:15.540063 2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-r7z7z\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="af130afb-9312-478a-a556-5cc2af4a8d99" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" Jul 18 01:56:15.542534 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:56:15.542374 2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-r7z7z\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="af130afb-9312-478a-a556-5cc2af4a8d99" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" Jul 18 01:56:15.544483 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:56:15.544451 2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wz68z\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z" Jul 18 01:56:15.544595 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.544519 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-696mp\" (UniqueName: \"kubernetes.io/projected/b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5-kube-api-access-696mp\") pod \"kuadrant-operator-controller-manager-86878c769f-x4f6p\" (UID: \"b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p" Jul 18 01:56:15.544665 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.544617 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x4f6p\" (UID: \"b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p" Jul 18 01:56:15.550490 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.550459 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5" path="/var/lib/kubelet/pods/b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5/volumes" Jul 18 01:56:15.645916 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.645878 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1792c44b-2956-4599-8a93-b1bee9ddfe82-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hfqkr\" (UID: \"1792c44b-2956-4599-8a93-b1bee9ddfe82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" Jul 18 01:56:15.646067 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.645925 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5sxq\" (UniqueName: \"kubernetes.io/projected/d60a599b-65ac-4991-b2e4-a74387659a39-kube-api-access-s5sxq\") pod \"kuadrant-operator-controller-manager-86878c769f-g6p78\" (UID: \"d60a599b-65ac-4991-b2e4-a74387659a39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" Jul 18 01:56:15.646127 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.646074 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x4f6p\" (UID: \"b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p" Jul 18 01:56:15.646171 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.646143 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw9vr\" (UniqueName: \"kubernetes.io/projected/1792c44b-2956-4599-8a93-b1bee9ddfe82-kube-api-access-xw9vr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hfqkr\" (UID: \"1792c44b-2956-4599-8a93-b1bee9ddfe82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" Jul 18 01:56:15.646204 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.646189 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-696mp\" (UniqueName: \"kubernetes.io/projected/b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5-kube-api-access-696mp\") pod \"kuadrant-operator-controller-manager-86878c769f-x4f6p\" (UID: \"b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p" Jul 18 01:56:15.646253 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.646237 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d60a599b-65ac-4991-b2e4-a74387659a39-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g6p78\" (UID: \"d60a599b-65ac-4991-b2e4-a74387659a39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" Jul 18 01:56:15.646634 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.646608 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-x4f6p\" (UID: \"b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p" Jul 18 01:56:15.650997 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:56:15.650975 2548 projected.go:196] Error preparing data for projected volume kube-api-access-696mp for pod kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p: failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-x4f6p" not found Jul 18 01:56:15.651117 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:56:15.651069 2548 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5-kube-api-access-696mp podName:b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5 nodeName:}" failed. No retries permitted until 2026-07-18 01:56:16.151046562 +0000 UTC m=+485.003789890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-696mp" (UniqueName: "kubernetes.io/projected/b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5-kube-api-access-696mp") pod "kuadrant-operator-controller-manager-86878c769f-x4f6p" (UID: "b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5") : failed to fetch token: pod "kuadrant-operator-controller-manager-86878c769f-x4f6p" not found Jul 18 01:56:15.747628 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.747532 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d60a599b-65ac-4991-b2e4-a74387659a39-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g6p78\" (UID: \"d60a599b-65ac-4991-b2e4-a74387659a39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" Jul 18 01:56:15.747628 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.747616 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1792c44b-2956-4599-8a93-b1bee9ddfe82-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hfqkr\" (UID: \"1792c44b-2956-4599-8a93-b1bee9ddfe82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" Jul 18 01:56:15.747897 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.747643 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5sxq\" (UniqueName: \"kubernetes.io/projected/d60a599b-65ac-4991-b2e4-a74387659a39-kube-api-access-s5sxq\") pod \"kuadrant-operator-controller-manager-86878c769f-g6p78\" (UID: \"d60a599b-65ac-4991-b2e4-a74387659a39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" Jul 18 01:56:15.747897 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.747737 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw9vr\" (UniqueName: \"kubernetes.io/projected/1792c44b-2956-4599-8a93-b1bee9ddfe82-kube-api-access-xw9vr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hfqkr\" (UID: \"1792c44b-2956-4599-8a93-b1bee9ddfe82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" Jul 18 01:56:15.748016 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.747996 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d60a599b-65ac-4991-b2e4-a74387659a39-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g6p78\" (UID: \"d60a599b-65ac-4991-b2e4-a74387659a39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" Jul 18 01:56:15.748128 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.748098 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1792c44b-2956-4599-8a93-b1bee9ddfe82-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hfqkr\" (UID: \"1792c44b-2956-4599-8a93-b1bee9ddfe82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" Jul 18 01:56:15.758278 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.758254 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw9vr\" (UniqueName: \"kubernetes.io/projected/1792c44b-2956-4599-8a93-b1bee9ddfe82-kube-api-access-xw9vr\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hfqkr\" (UID: \"1792c44b-2956-4599-8a93-b1bee9ddfe82\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" Jul 18 01:56:15.758422 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.758387 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5sxq\" (UniqueName: \"kubernetes.io/projected/d60a599b-65ac-4991-b2e4-a74387659a39-kube-api-access-s5sxq\") pod \"kuadrant-operator-controller-manager-86878c769f-g6p78\" (UID: \"d60a599b-65ac-4991-b2e4-a74387659a39\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" Jul 18 01:56:15.810657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.810614 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p" Jul 18 01:56:15.823729 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.823703 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p" Jul 18 01:56:15.825940 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:56:15.825909 2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-x4f6p\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p" Jul 18 01:56:15.886543 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.886498 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" Jul 18 01:56:15.893944 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.893917 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" Jul 18 01:56:15.949766 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.949735 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5-extensions-socket-volume\") pod \"b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5\" (UID: \"b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5\") " Jul 18 01:56:15.950016 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.949996 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-696mp\" (UniqueName: \"kubernetes.io/projected/b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5-kube-api-access-696mp\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:56:15.950136 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:15.950009 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5" (UID: "b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:56:16.051340 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:16.051272 2548 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5-extensions-socket-volume\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:56:16.815628 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:16.815596 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p" Jul 18 01:56:16.817694 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:56:16.817664 2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-x4f6p\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p" Jul 18 01:56:16.825571 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:56:16.825542 2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-x4f6p\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-x4f6p" Jul 18 01:56:17.549710 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:17.549674 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5" path="/var/lib/kubelet/pods/b98c0b9a-bee5-4d81-83ac-35f7ba1cd3f5/volumes" Jul 18 01:56:18.775214 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:18.775108 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x7bs7" Jul 18 01:56:27.183499 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.183468 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z" Jul 18 01:56:27.185530 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:56:27.185494 2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wz68z\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z" Jul 18 01:56:27.189129 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.189109 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" Jul 18 01:56:27.191000 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:56:27.190974 2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wz68z\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z" Jul 18 01:56:27.192728 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:56:27.192700 2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-r7z7z\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="af130afb-9312-478a-a556-5cc2af4a8d99" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" Jul 18 01:56:27.260244 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.260166 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af130afb-9312-478a-a556-5cc2af4a8d99-extensions-socket-volume\") pod \"af130afb-9312-478a-a556-5cc2af4a8d99\" (UID: \"af130afb-9312-478a-a556-5cc2af4a8d99\") " Jul 18 01:56:27.260244 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.260213 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8tfl\" (UniqueName: \"kubernetes.io/projected/af130afb-9312-478a-a556-5cc2af4a8d99-kube-api-access-s8tfl\") pod \"af130afb-9312-478a-a556-5cc2af4a8d99\" (UID: \"af130afb-9312-478a-a556-5cc2af4a8d99\") " Jul 18 01:56:27.260444 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.260258 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lf5pw\" (UniqueName: \"kubernetes.io/projected/ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd-kube-api-access-lf5pw\") pod \"ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd\" (UID: \"ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd\") " Jul 18 01:56:27.260649 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.260622 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af130afb-9312-478a-a556-5cc2af4a8d99-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "af130afb-9312-478a-a556-5cc2af4a8d99" (UID: "af130afb-9312-478a-a556-5cc2af4a8d99"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:56:27.262358 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.262332 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af130afb-9312-478a-a556-5cc2af4a8d99-kube-api-access-s8tfl" (OuterVolumeSpecName: "kube-api-access-s8tfl") pod "af130afb-9312-478a-a556-5cc2af4a8d99" (UID: "af130afb-9312-478a-a556-5cc2af4a8d99"). InnerVolumeSpecName "kube-api-access-s8tfl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:56:27.262469 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.262448 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd-kube-api-access-lf5pw" (OuterVolumeSpecName: "kube-api-access-lf5pw") pod "ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd" (UID: "ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd"). InnerVolumeSpecName "kube-api-access-lf5pw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:56:27.361069 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.361034 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lf5pw\" (UniqueName: \"kubernetes.io/projected/ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd-kube-api-access-lf5pw\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:56:27.361069 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.361061 2548 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/af130afb-9312-478a-a556-5cc2af4a8d99-extensions-socket-volume\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:56:27.361069 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.361071 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8tfl\" (UniqueName: \"kubernetes.io/projected/af130afb-9312-478a-a556-5cc2af4a8d99-kube-api-access-s8tfl\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:56:27.395510 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.395477 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78"] Jul 18 01:56:27.397383 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:56:27.397334 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd60a599b_65ac_4991_b2e4_a74387659a39.slice/crio-44ea22744b31d172825373dcd6742ecf50cf74cb587850a43918464fedd400bb WatchSource:0}: Error finding container 44ea22744b31d172825373dcd6742ecf50cf74cb587850a43918464fedd400bb: Status 404 returned error can't find the container with id 44ea22744b31d172825373dcd6742ecf50cf74cb587850a43918464fedd400bb Jul 18 01:56:27.397835 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.397813 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr"] Jul 18 01:56:27.400741 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:56:27.400720 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1792c44b_2956_4599_8a93_b1bee9ddfe82.slice/crio-29518f2da95a27c8cfef434e5e55e8cd8a0f7f416ac57de5f158899ca9590d2b WatchSource:0}: Error finding container 29518f2da95a27c8cfef434e5e55e8cd8a0f7f416ac57de5f158899ca9590d2b: Status 404 returned error can't find the container with id 29518f2da95a27c8cfef434e5e55e8cd8a0f7f416ac57de5f158899ca9590d2b Jul 18 01:56:27.549868 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.549761 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af130afb-9312-478a-a556-5cc2af4a8d99" path="/var/lib/kubelet/pods/af130afb-9312-478a-a556-5cc2af4a8d99/volumes" Jul 18 01:56:27.550179 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.550160 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd" path="/var/lib/kubelet/pods/ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd/volumes" Jul 18 01:56:27.871213 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.871118 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" event={"ID":"d60a599b-65ac-4991-b2e4-a74387659a39","Type":"ContainerStarted","Data":"482effd2549a36a3633ce7f31b5ed579441eb9c2b95a8cea046a79e92e8aae01"} Jul 18 01:56:27.871213 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.871177 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" event={"ID":"d60a599b-65ac-4991-b2e4-a74387659a39","Type":"ContainerStarted","Data":"44ea22744b31d172825373dcd6742ecf50cf74cb587850a43918464fedd400bb"} Jul 18 01:56:27.871431 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.871223 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" Jul 18 01:56:27.872473 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.872445 2548 generic.go:356] "Generic (PLEG): container finished" podID="af130afb-9312-478a-a556-5cc2af4a8d99" containerID="600abf58f867fbb1f69e7c756f3daa179bdeba93d656872f233998d4c2881aa4" exitCode=0 Jul 18 01:56:27.872581 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.872485 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" Jul 18 01:56:27.872581 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.872536 2548 scope.go:117] "RemoveContainer" containerID="600abf58f867fbb1f69e7c756f3daa179bdeba93d656872f233998d4c2881aa4" Jul 18 01:56:27.873694 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.873668 2548 generic.go:356] "Generic (PLEG): container finished" podID="ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd" containerID="5a29105182effb3e85bc788425dc97396e12aa3a96b1ddfaecde97aafb8de44c" exitCode=0 Jul 18 01:56:27.873832 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.873714 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z" Jul 18 01:56:27.875556 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.875530 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9nfft" event={"ID":"806dc654-0543-4b3b-baec-68d01a5db966","Type":"ContainerStarted","Data":"29492c91120474cdfa7a6ee23c28600aa0b4e24d35a099f11e1eb7ca2a52087f"} Jul 18 01:56:27.877372 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.877351 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" event={"ID":"1792c44b-2956-4599-8a93-b1bee9ddfe82","Type":"ContainerStarted","Data":"ec7ae1f7e6e7b88f258a4b65a0252999d823f9882c8dfefab1e96462881cfdc2"} Jul 18 01:56:27.877466 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.877387 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" event={"ID":"1792c44b-2956-4599-8a93-b1bee9ddfe82","Type":"ContainerStarted","Data":"29518f2da95a27c8cfef434e5e55e8cd8a0f7f416ac57de5f158899ca9590d2b"} Jul 18 01:56:27.877466 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.877407 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" Jul 18 01:56:27.881797 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.881768 2548 scope.go:117] "RemoveContainer" containerID="600abf58f867fbb1f69e7c756f3daa179bdeba93d656872f233998d4c2881aa4" Jul 18 01:56:27.882053 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:56:27.882032 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"600abf58f867fbb1f69e7c756f3daa179bdeba93d656872f233998d4c2881aa4\": container with ID starting with 600abf58f867fbb1f69e7c756f3daa179bdeba93d656872f233998d4c2881aa4 not found: ID does not exist" containerID="600abf58f867fbb1f69e7c756f3daa179bdeba93d656872f233998d4c2881aa4" Jul 18 01:56:27.882147 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.882057 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"600abf58f867fbb1f69e7c756f3daa179bdeba93d656872f233998d4c2881aa4"} err="failed to get container status \"600abf58f867fbb1f69e7c756f3daa179bdeba93d656872f233998d4c2881aa4\": rpc error: code = NotFound desc = could not find container \"600abf58f867fbb1f69e7c756f3daa179bdeba93d656872f233998d4c2881aa4\": container with ID starting with 600abf58f867fbb1f69e7c756f3daa179bdeba93d656872f233998d4c2881aa4 not found: ID does not exist" Jul 18 01:56:27.882147 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.882073 2548 scope.go:117] "RemoveContainer" containerID="5a29105182effb3e85bc788425dc97396e12aa3a96b1ddfaecde97aafb8de44c" Jul 18 01:56:27.889458 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.889441 2548 scope.go:117] "RemoveContainer" containerID="5a29105182effb3e85bc788425dc97396e12aa3a96b1ddfaecde97aafb8de44c" Jul 18 01:56:27.889696 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:56:27.889670 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a29105182effb3e85bc788425dc97396e12aa3a96b1ddfaecde97aafb8de44c\": container with ID starting with 5a29105182effb3e85bc788425dc97396e12aa3a96b1ddfaecde97aafb8de44c not found: ID does not exist" containerID="5a29105182effb3e85bc788425dc97396e12aa3a96b1ddfaecde97aafb8de44c" Jul 18 01:56:27.889745 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.889696 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a29105182effb3e85bc788425dc97396e12aa3a96b1ddfaecde97aafb8de44c"} err="failed to get container status \"5a29105182effb3e85bc788425dc97396e12aa3a96b1ddfaecde97aafb8de44c\": rpc error: code = NotFound desc = could not find container \"5a29105182effb3e85bc788425dc97396e12aa3a96b1ddfaecde97aafb8de44c\": container with ID starting with 5a29105182effb3e85bc788425dc97396e12aa3a96b1ddfaecde97aafb8de44c not found: ID does not exist" Jul 18 01:56:27.893632 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.893594 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" podStartSLOduration=12.893579447 podStartE2EDuration="12.893579447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:27.892364927 +0000 UTC m=+496.745108272" watchObservedRunningTime="2026-07-18 01:56:27.893579447 +0000 UTC m=+496.746322792" Jul 18 01:56:27.894319 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:56:27.894282 2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-r7z7z\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="af130afb-9312-478a-a556-5cc2af4a8d99" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r7z7z" Jul 18 01:56:27.896073 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:56:27.896050 2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-wz68z\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-wz68z" Jul 18 01:56:27.913230 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.913187 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-9nfft" podStartSLOduration=1.532351175 podStartE2EDuration="22.913173632s" totalImagesPullingTime="21.380822457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:27.912521069 +0000 UTC m=+496.765264414" watchObservedRunningTime="2026-07-18 01:56:27.913173632 +0000 UTC m=+496.765916976" Jul 18 01:56:27.932972 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:27.932926 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" podStartSLOduration=12.932909877 podStartE2EDuration="12.932909877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:27.931809016 +0000 UTC m=+496.784552362" watchObservedRunningTime="2026-07-18 01:56:27.932909877 +0000 UTC m=+496.785653221" Jul 18 01:56:38.886994 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:38.886960 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" Jul 18 01:56:38.887423 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:38.887124 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" Jul 18 01:56:38.960464 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:38.960435 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr"] Jul 18 01:56:38.960683 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:38.960659 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" podUID="1792c44b-2956-4599-8a93-b1bee9ddfe82" containerName="manager" containerID="cri-o://ec7ae1f7e6e7b88f258a4b65a0252999d823f9882c8dfefab1e96462881cfdc2" gracePeriod=10 Jul 18 01:56:39.715202 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:39.715174 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" Jul 18 01:56:39.766183 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:39.766156 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1792c44b-2956-4599-8a93-b1bee9ddfe82-extensions-socket-volume\") pod \"1792c44b-2956-4599-8a93-b1bee9ddfe82\" (UID: \"1792c44b-2956-4599-8a93-b1bee9ddfe82\") " Jul 18 01:56:39.766351 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:39.766195 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xw9vr\" (UniqueName: \"kubernetes.io/projected/1792c44b-2956-4599-8a93-b1bee9ddfe82-kube-api-access-xw9vr\") pod \"1792c44b-2956-4599-8a93-b1bee9ddfe82\" (UID: \"1792c44b-2956-4599-8a93-b1bee9ddfe82\") " Jul 18 01:56:39.766610 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:39.766580 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1792c44b-2956-4599-8a93-b1bee9ddfe82-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1792c44b-2956-4599-8a93-b1bee9ddfe82" (UID: "1792c44b-2956-4599-8a93-b1bee9ddfe82"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 01:56:39.768285 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:39.768263 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1792c44b-2956-4599-8a93-b1bee9ddfe82-kube-api-access-xw9vr" (OuterVolumeSpecName: "kube-api-access-xw9vr") pod "1792c44b-2956-4599-8a93-b1bee9ddfe82" (UID: "1792c44b-2956-4599-8a93-b1bee9ddfe82"). InnerVolumeSpecName "kube-api-access-xw9vr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:56:39.867482 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:39.867403 2548 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1792c44b-2956-4599-8a93-b1bee9ddfe82-extensions-socket-volume\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:56:39.867482 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:39.867426 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw9vr\" (UniqueName: \"kubernetes.io/projected/1792c44b-2956-4599-8a93-b1bee9ddfe82-kube-api-access-xw9vr\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:56:39.938307 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:39.938272 2548 generic.go:356] "Generic (PLEG): container finished" podID="1792c44b-2956-4599-8a93-b1bee9ddfe82" containerID="ec7ae1f7e6e7b88f258a4b65a0252999d823f9882c8dfefab1e96462881cfdc2" exitCode=0 Jul 18 01:56:39.938818 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:39.938321 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" Jul 18 01:56:39.938818 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:39.938359 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" event={"ID":"1792c44b-2956-4599-8a93-b1bee9ddfe82","Type":"ContainerDied","Data":"ec7ae1f7e6e7b88f258a4b65a0252999d823f9882c8dfefab1e96462881cfdc2"} Jul 18 01:56:39.938818 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:39.938413 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr" event={"ID":"1792c44b-2956-4599-8a93-b1bee9ddfe82","Type":"ContainerDied","Data":"29518f2da95a27c8cfef434e5e55e8cd8a0f7f416ac57de5f158899ca9590d2b"} Jul 18 01:56:39.938818 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:39.938438 2548 scope.go:117] "RemoveContainer" containerID="ec7ae1f7e6e7b88f258a4b65a0252999d823f9882c8dfefab1e96462881cfdc2" Jul 18 01:56:39.947172 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:39.947156 2548 scope.go:117] "RemoveContainer" containerID="ec7ae1f7e6e7b88f258a4b65a0252999d823f9882c8dfefab1e96462881cfdc2" Jul 18 01:56:39.947428 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:56:39.947411 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec7ae1f7e6e7b88f258a4b65a0252999d823f9882c8dfefab1e96462881cfdc2\": container with ID starting with ec7ae1f7e6e7b88f258a4b65a0252999d823f9882c8dfefab1e96462881cfdc2 not found: ID does not exist" containerID="ec7ae1f7e6e7b88f258a4b65a0252999d823f9882c8dfefab1e96462881cfdc2" Jul 18 01:56:39.947466 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:39.947439 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec7ae1f7e6e7b88f258a4b65a0252999d823f9882c8dfefab1e96462881cfdc2"} err="failed to get container status \"ec7ae1f7e6e7b88f258a4b65a0252999d823f9882c8dfefab1e96462881cfdc2\": rpc error: code = NotFound desc = could not find container \"ec7ae1f7e6e7b88f258a4b65a0252999d823f9882c8dfefab1e96462881cfdc2\": container with ID starting with ec7ae1f7e6e7b88f258a4b65a0252999d823f9882c8dfefab1e96462881cfdc2 not found: ID does not exist" Jul 18 01:56:39.963661 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:39.963637 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr"] Jul 18 01:56:39.967727 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:39.967700 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hfqkr"] Jul 18 01:56:41.548600 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:41.548567 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1792c44b-2956-4599-8a93-b1bee9ddfe82" path="/var/lib/kubelet/pods/1792c44b-2956-4599-8a93-b1bee9ddfe82/volumes" Jul 18 01:56:55.328277 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.328246 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz"] Jul 18 01:56:55.328648 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.328615 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd" containerName="manager" Jul 18 01:56:55.328648 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.328627 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd" containerName="manager" Jul 18 01:56:55.328648 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.328643 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1792c44b-2956-4599-8a93-b1bee9ddfe82" containerName="manager" Jul 18 01:56:55.328648 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.328649 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="1792c44b-2956-4599-8a93-b1bee9ddfe82" containerName="manager" Jul 18 01:56:55.328806 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.328711 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca74ce5f-b28e-4f09-a63a-c9a88ab1a3dd" containerName="manager" Jul 18 01:56:55.328806 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.328722 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="1792c44b-2956-4599-8a93-b1bee9ddfe82" containerName="manager" Jul 18 01:56:55.335150 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.335133 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.339108 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.339077 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 18 01:56:55.339248 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.339091 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-x222m\"" Jul 18 01:56:55.351015 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.350985 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz"] Jul 18 01:56:55.396150 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.396122 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.396322 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.396169 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlwpr\" (UniqueName: \"kubernetes.io/projected/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-kube-api-access-nlwpr\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.396322 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.396197 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.396322 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.396282 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.396439 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.396325 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.396439 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.396350 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.396439 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.396380 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.396583 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.396476 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.396583 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.396548 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.497247 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.497203 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.497438 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.497264 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.497438 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.497296 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.497438 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.497338 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlwpr\" (UniqueName: \"kubernetes.io/projected/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-kube-api-access-nlwpr\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.497438 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.497368 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.497438 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.497403 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.497438 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.497427 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.497743 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.497449 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.497743 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.497479 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.497743 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.497689 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.498050 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.498026 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.498170 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.498117 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.498219 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.498185 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.498589 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.498553 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.499893 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.499873 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.500191 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.500174 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.506280 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.506257 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.506543 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.506521 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlwpr\" (UniqueName: \"kubernetes.io/projected/aeddc1c9-3b8b-4f5b-91a8-ac2655950649-kube-api-access-nlwpr\") pod \"maas-default-gateway-openshift-default-56654dcf7-62vpz\" (UID: \"aeddc1c9-3b8b-4f5b-91a8-ac2655950649\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.645897 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.645809 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:55.808676 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:55.808651 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz"] Jul 18 01:56:55.810947 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:56:55.810910 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeddc1c9_3b8b_4f5b_91a8_ac2655950649.slice/crio-2cfde510affc7062f79ab40bd43baead5e8c94b9f3f6d02137ed8f5a6ed04907 WatchSource:0}: Error finding container 2cfde510affc7062f79ab40bd43baead5e8c94b9f3f6d02137ed8f5a6ed04907: Status 404 returned error can't find the container with id 2cfde510affc7062f79ab40bd43baead5e8c94b9f3f6d02137ed8f5a6ed04907 Jul 18 01:56:56.017205 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:56.017171 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" event={"ID":"aeddc1c9-3b8b-4f5b-91a8-ac2655950649","Type":"ContainerStarted","Data":"2cfde510affc7062f79ab40bd43baead5e8c94b9f3f6d02137ed8f5a6ed04907"} Jul 18 01:56:58.484042 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:58.484010 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:56:58.484299 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:58.484079 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:56:58.484299 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:58.484109 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:56:59.033573 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:59.033540 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" event={"ID":"aeddc1c9-3b8b-4f5b-91a8-ac2655950649","Type":"ContainerStarted","Data":"39784ed588e18351ae2ca964e020bcb3485f37b0b4f7d1d88dcfca0d2e2b8530"} Jul 18 01:56:59.054369 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:59.054322 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" podStartSLOduration=1.383548223 podStartE2EDuration="4.054305952s" totalImagesPullingTime="2.670757729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:59.052465172 +0000 UTC m=+527.905208517" watchObservedRunningTime="2026-07-18 01:56:59.054305952 +0000 UTC m=+527.907049295" Jul 18 01:56:59.646028 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:59.645999 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:56:59.651356 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:56:59.651317 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:57:00.032383 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.032350 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ff5nd"] Jul 18 01:57:00.035815 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.035789 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" Jul 18 01:57:00.038991 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.038970 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:57:00.039377 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.039361 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 18 01:57:00.039943 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.039925 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:57:00.049203 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.049181 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ff5nd"] Jul 18 01:57:00.124809 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.124764 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ff5nd"] Jul 18 01:57:00.142144 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.142118 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2429aa3e-046f-4ac9-b5d0-228476008ca6-config-file\") pod \"limitador-limitador-64c94fd64f-ff5nd\" (UID: \"2429aa3e-046f-4ac9-b5d0-228476008ca6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" Jul 18 01:57:00.142286 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.142266 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbz4r\" (UniqueName: \"kubernetes.io/projected/2429aa3e-046f-4ac9-b5d0-228476008ca6-kube-api-access-wbz4r\") pod \"limitador-limitador-64c94fd64f-ff5nd\" (UID: \"2429aa3e-046f-4ac9-b5d0-228476008ca6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" Jul 18 01:57:00.243688 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.243657 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbz4r\" (UniqueName: \"kubernetes.io/projected/2429aa3e-046f-4ac9-b5d0-228476008ca6-kube-api-access-wbz4r\") pod \"limitador-limitador-64c94fd64f-ff5nd\" (UID: \"2429aa3e-046f-4ac9-b5d0-228476008ca6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" Jul 18 01:57:00.243890 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.243711 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2429aa3e-046f-4ac9-b5d0-228476008ca6-config-file\") pod \"limitador-limitador-64c94fd64f-ff5nd\" (UID: \"2429aa3e-046f-4ac9-b5d0-228476008ca6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" Jul 18 01:57:00.244300 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.244282 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2429aa3e-046f-4ac9-b5d0-228476008ca6-config-file\") pod \"limitador-limitador-64c94fd64f-ff5nd\" (UID: \"2429aa3e-046f-4ac9-b5d0-228476008ca6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" Jul 18 01:57:00.251876 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.251847 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbz4r\" (UniqueName: \"kubernetes.io/projected/2429aa3e-046f-4ac9-b5d0-228476008ca6-kube-api-access-wbz4r\") pod \"limitador-limitador-64c94fd64f-ff5nd\" (UID: \"2429aa3e-046f-4ac9-b5d0-228476008ca6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" Jul 18 01:57:00.346277 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.346210 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" Jul 18 01:57:00.479864 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.479836 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ff5nd"] Jul 18 01:57:00.481727 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:57:00.481700 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2429aa3e_046f_4ac9_b5d0_228476008ca6.slice/crio-a79895615737d8bf1a47d4e36ac2f07cd0b0ff0261fded95be638f0481ee6f65 WatchSource:0}: Error finding container a79895615737d8bf1a47d4e36ac2f07cd0b0ff0261fded95be638f0481ee6f65: Status 404 returned error can't find the container with id a79895615737d8bf1a47d4e36ac2f07cd0b0ff0261fded95be638f0481ee6f65 Jul 18 01:57:00.811717 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.811680 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-phk9k"] Jul 18 01:57:00.816884 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.816858 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-phk9k" Jul 18 01:57:00.818950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.818931 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-dblmg\"" Jul 18 01:57:00.823358 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.823335 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-phk9k"] Jul 18 01:57:00.916563 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.916526 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-72s9r"] Jul 18 01:57:00.919773 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.919756 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-72s9r" Jul 18 01:57:00.923826 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.923801 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-72s9r"] Jul 18 01:57:00.948126 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:00.948093 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vkj5\" (UniqueName: \"kubernetes.io/projected/c47e4c7a-4040-4e47-b557-c2b12c3ef2dd-kube-api-access-5vkj5\") pod \"authorino-58cd955675-phk9k\" (UID: \"c47e4c7a-4040-4e47-b557-c2b12c3ef2dd\") " pod="kuadrant-system/authorino-58cd955675-phk9k" Jul 18 01:57:01.044667 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:01.044633 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" event={"ID":"2429aa3e-046f-4ac9-b5d0-228476008ca6","Type":"ContainerStarted","Data":"a79895615737d8bf1a47d4e36ac2f07cd0b0ff0261fded95be638f0481ee6f65"} Jul 18 01:57:01.049384 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:01.049360 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkf4k\" (UniqueName: \"kubernetes.io/projected/07a70a9a-da9c-4e13-b7c0-71041c2d56a4-kube-api-access-zkf4k\") pod \"authorino-75d556bd77-72s9r\" (UID: \"07a70a9a-da9c-4e13-b7c0-71041c2d56a4\") " pod="kuadrant-system/authorino-75d556bd77-72s9r" Jul 18 01:57:01.049520 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:01.049396 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vkj5\" (UniqueName: \"kubernetes.io/projected/c47e4c7a-4040-4e47-b557-c2b12c3ef2dd-kube-api-access-5vkj5\") pod \"authorino-58cd955675-phk9k\" (UID: \"c47e4c7a-4040-4e47-b557-c2b12c3ef2dd\") " pod="kuadrant-system/authorino-58cd955675-phk9k" Jul 18 01:57:01.056846 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:01.056820 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vkj5\" (UniqueName: \"kubernetes.io/projected/c47e4c7a-4040-4e47-b557-c2b12c3ef2dd-kube-api-access-5vkj5\") pod \"authorino-58cd955675-phk9k\" (UID: \"c47e4c7a-4040-4e47-b557-c2b12c3ef2dd\") " pod="kuadrant-system/authorino-58cd955675-phk9k" Jul 18 01:57:01.127076 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:01.126998 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-phk9k" Jul 18 01:57:01.151505 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:01.150549 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkf4k\" (UniqueName: \"kubernetes.io/projected/07a70a9a-da9c-4e13-b7c0-71041c2d56a4-kube-api-access-zkf4k\") pod \"authorino-75d556bd77-72s9r\" (UID: \"07a70a9a-da9c-4e13-b7c0-71041c2d56a4\") " pod="kuadrant-system/authorino-75d556bd77-72s9r" Jul 18 01:57:01.159533 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:01.159477 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkf4k\" (UniqueName: \"kubernetes.io/projected/07a70a9a-da9c-4e13-b7c0-71041c2d56a4-kube-api-access-zkf4k\") pod \"authorino-75d556bd77-72s9r\" (UID: \"07a70a9a-da9c-4e13-b7c0-71041c2d56a4\") " pod="kuadrant-system/authorino-75d556bd77-72s9r" Jul 18 01:57:01.229806 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:01.229661 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-72s9r" Jul 18 01:57:01.327679 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:01.327651 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-phk9k"] Jul 18 01:57:01.331651 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:57:01.331623 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc47e4c7a_4040_4e47_b557_c2b12c3ef2dd.slice/crio-a36c2c5fe1ea587d94953d1e1035c81e93c90ab5b050e95f85a8b764d206baf6 WatchSource:0}: Error finding container a36c2c5fe1ea587d94953d1e1035c81e93c90ab5b050e95f85a8b764d206baf6: Status 404 returned error can't find the container with id a36c2c5fe1ea587d94953d1e1035c81e93c90ab5b050e95f85a8b764d206baf6 Jul 18 01:57:01.429427 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:01.429393 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-72s9r"] Jul 18 01:57:01.433720 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:57:01.433692 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07a70a9a_da9c_4e13_b7c0_71041c2d56a4.slice/crio-87699cdcee4ed2c2c711e6549b55aa6de629d3bff0710a41aeb12a0e69a9278b WatchSource:0}: Error finding container 87699cdcee4ed2c2c711e6549b55aa6de629d3bff0710a41aeb12a0e69a9278b: Status 404 returned error can't find the container with id 87699cdcee4ed2c2c711e6549b55aa6de629d3bff0710a41aeb12a0e69a9278b Jul 18 01:57:02.073270 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:02.073225 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-phk9k" event={"ID":"c47e4c7a-4040-4e47-b557-c2b12c3ef2dd","Type":"ContainerStarted","Data":"a36c2c5fe1ea587d94953d1e1035c81e93c90ab5b050e95f85a8b764d206baf6"} Jul 18 01:57:02.078386 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:02.078352 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-72s9r" event={"ID":"07a70a9a-da9c-4e13-b7c0-71041c2d56a4","Type":"ContainerStarted","Data":"87699cdcee4ed2c2c711e6549b55aa6de629d3bff0710a41aeb12a0e69a9278b"} Jul 18 01:57:06.104211 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:06.104108 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" event={"ID":"2429aa3e-046f-4ac9-b5d0-228476008ca6","Type":"ContainerStarted","Data":"33a076e51fd53cf9c9e004c773575e2d4d7d132a5ca62d5d85c5ecc40235e016"} Jul 18 01:57:06.104605 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:06.104274 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" Jul 18 01:57:06.105668 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:06.105639 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-phk9k" event={"ID":"c47e4c7a-4040-4e47-b557-c2b12c3ef2dd","Type":"ContainerStarted","Data":"c3a10befa35ba14cea2cf7cd4a725edb3a7db1706c94eda8c781572c07aed33b"} Jul 18 01:57:06.107129 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:06.107107 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-72s9r" event={"ID":"07a70a9a-da9c-4e13-b7c0-71041c2d56a4","Type":"ContainerStarted","Data":"e1f948933f4368e89fcbecd342670a40bee17487839d6131d1c471fe2718d6c8"} Jul 18 01:57:06.120542 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:06.120504 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" podStartSLOduration=0.877333101 podStartE2EDuration="6.120491178s" totalImagesPullingTime="5.243158077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:06.118849863 +0000 UTC m=+534.971593206" watchObservedRunningTime="2026-07-18 01:57:06.120491178 +0000 UTC m=+534.973234522" Jul 18 01:57:06.132743 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:06.132705 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-phk9k" podStartSLOduration=1.795109365 podStartE2EDuration="6.132695457s" totalImagesPullingTime="4.337586092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:06.131672696 +0000 UTC m=+534.984416055" watchObservedRunningTime="2026-07-18 01:57:06.132695457 +0000 UTC m=+534.985438800" Jul 18 01:57:06.148030 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:06.147992 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-72s9r" podStartSLOduration=1.872223696 podStartE2EDuration="6.147978266s" totalImagesPullingTime="4.27575457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:06.146021972 +0000 UTC m=+534.998765319" watchObservedRunningTime="2026-07-18 01:57:06.147978266 +0000 UTC m=+535.000721611" Jul 18 01:57:06.173742 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:06.173713 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-phk9k"] Jul 18 01:57:08.116598 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:08.116552 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-phk9k" podUID="c47e4c7a-4040-4e47-b557-c2b12c3ef2dd" containerName="authorino" containerID="cri-o://c3a10befa35ba14cea2cf7cd4a725edb3a7db1706c94eda8c781572c07aed33b" gracePeriod=30 Jul 18 01:57:08.359788 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:08.359753 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-phk9k" Jul 18 01:57:08.522350 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:08.522322 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vkj5\" (UniqueName: \"kubernetes.io/projected/c47e4c7a-4040-4e47-b557-c2b12c3ef2dd-kube-api-access-5vkj5\") pod \"c47e4c7a-4040-4e47-b557-c2b12c3ef2dd\" (UID: \"c47e4c7a-4040-4e47-b557-c2b12c3ef2dd\") " Jul 18 01:57:08.524261 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:08.524238 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c47e4c7a-4040-4e47-b557-c2b12c3ef2dd-kube-api-access-5vkj5" (OuterVolumeSpecName: "kube-api-access-5vkj5") pod "c47e4c7a-4040-4e47-b557-c2b12c3ef2dd" (UID: "c47e4c7a-4040-4e47-b557-c2b12c3ef2dd"). InnerVolumeSpecName "kube-api-access-5vkj5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:57:08.623696 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:08.623670 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vkj5\" (UniqueName: \"kubernetes.io/projected/c47e4c7a-4040-4e47-b557-c2b12c3ef2dd-kube-api-access-5vkj5\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:57:09.122436 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:09.122401 2548 generic.go:356] "Generic (PLEG): container finished" podID="c47e4c7a-4040-4e47-b557-c2b12c3ef2dd" containerID="c3a10befa35ba14cea2cf7cd4a725edb3a7db1706c94eda8c781572c07aed33b" exitCode=0 Jul 18 01:57:09.122852 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:09.122444 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-phk9k" Jul 18 01:57:09.122852 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:09.122483 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-phk9k" event={"ID":"c47e4c7a-4040-4e47-b557-c2b12c3ef2dd","Type":"ContainerDied","Data":"c3a10befa35ba14cea2cf7cd4a725edb3a7db1706c94eda8c781572c07aed33b"} Jul 18 01:57:09.122852 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:09.122526 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-phk9k" event={"ID":"c47e4c7a-4040-4e47-b557-c2b12c3ef2dd","Type":"ContainerDied","Data":"a36c2c5fe1ea587d94953d1e1035c81e93c90ab5b050e95f85a8b764d206baf6"} Jul 18 01:57:09.122852 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:09.122543 2548 scope.go:117] "RemoveContainer" containerID="c3a10befa35ba14cea2cf7cd4a725edb3a7db1706c94eda8c781572c07aed33b" Jul 18 01:57:09.131559 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:09.131541 2548 scope.go:117] "RemoveContainer" containerID="c3a10befa35ba14cea2cf7cd4a725edb3a7db1706c94eda8c781572c07aed33b" Jul 18 01:57:09.131846 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:57:09.131823 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3a10befa35ba14cea2cf7cd4a725edb3a7db1706c94eda8c781572c07aed33b\": container with ID starting with c3a10befa35ba14cea2cf7cd4a725edb3a7db1706c94eda8c781572c07aed33b not found: ID does not exist" containerID="c3a10befa35ba14cea2cf7cd4a725edb3a7db1706c94eda8c781572c07aed33b" Jul 18 01:57:09.131949 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:09.131857 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3a10befa35ba14cea2cf7cd4a725edb3a7db1706c94eda8c781572c07aed33b"} err="failed to get container status \"c3a10befa35ba14cea2cf7cd4a725edb3a7db1706c94eda8c781572c07aed33b\": rpc error: code = NotFound desc = could not find container \"c3a10befa35ba14cea2cf7cd4a725edb3a7db1706c94eda8c781572c07aed33b\": container with ID starting with c3a10befa35ba14cea2cf7cd4a725edb3a7db1706c94eda8c781572c07aed33b not found: ID does not exist" Jul 18 01:57:09.143881 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:09.143862 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-phk9k"] Jul 18 01:57:09.148255 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:09.148231 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-phk9k"] Jul 18 01:57:09.549033 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:09.548994 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c47e4c7a-4040-4e47-b557-c2b12c3ef2dd" path="/var/lib/kubelet/pods/c47e4c7a-4040-4e47-b557-c2b12c3ef2dd/volumes" Jul 18 01:57:14.376239 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:14.376202 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ff5nd"] Jul 18 01:57:14.376710 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:14.376391 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" podUID="2429aa3e-046f-4ac9-b5d0-228476008ca6" containerName="limitador" containerID="cri-o://33a076e51fd53cf9c9e004c773575e2d4d7d132a5ca62d5d85c5ecc40235e016" gracePeriod=30 Jul 18 01:57:14.377113 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:14.377096 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" Jul 18 01:57:14.928642 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:14.928616 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" Jul 18 01:57:14.979245 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:14.979179 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbz4r\" (UniqueName: \"kubernetes.io/projected/2429aa3e-046f-4ac9-b5d0-228476008ca6-kube-api-access-wbz4r\") pod \"2429aa3e-046f-4ac9-b5d0-228476008ca6\" (UID: \"2429aa3e-046f-4ac9-b5d0-228476008ca6\") " Jul 18 01:57:14.979245 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:14.979215 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2429aa3e-046f-4ac9-b5d0-228476008ca6-config-file\") pod \"2429aa3e-046f-4ac9-b5d0-228476008ca6\" (UID: \"2429aa3e-046f-4ac9-b5d0-228476008ca6\") " Jul 18 01:57:14.979614 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:14.979591 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2429aa3e-046f-4ac9-b5d0-228476008ca6-config-file" (OuterVolumeSpecName: "config-file") pod "2429aa3e-046f-4ac9-b5d0-228476008ca6" (UID: "2429aa3e-046f-4ac9-b5d0-228476008ca6"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 01:57:14.981177 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:14.981155 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2429aa3e-046f-4ac9-b5d0-228476008ca6-kube-api-access-wbz4r" (OuterVolumeSpecName: "kube-api-access-wbz4r") pod "2429aa3e-046f-4ac9-b5d0-228476008ca6" (UID: "2429aa3e-046f-4ac9-b5d0-228476008ca6"). InnerVolumeSpecName "kube-api-access-wbz4r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:57:15.080219 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:15.080197 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbz4r\" (UniqueName: \"kubernetes.io/projected/2429aa3e-046f-4ac9-b5d0-228476008ca6-kube-api-access-wbz4r\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:57:15.080219 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:15.080218 2548 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2429aa3e-046f-4ac9-b5d0-228476008ca6-config-file\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:57:15.152826 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:15.152797 2548 generic.go:356] "Generic (PLEG): container finished" podID="2429aa3e-046f-4ac9-b5d0-228476008ca6" containerID="33a076e51fd53cf9c9e004c773575e2d4d7d132a5ca62d5d85c5ecc40235e016" exitCode=0 Jul 18 01:57:15.152969 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:15.152844 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" Jul 18 01:57:15.152969 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:15.152870 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" event={"ID":"2429aa3e-046f-4ac9-b5d0-228476008ca6","Type":"ContainerDied","Data":"33a076e51fd53cf9c9e004c773575e2d4d7d132a5ca62d5d85c5ecc40235e016"} Jul 18 01:57:15.152969 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:15.152923 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ff5nd" event={"ID":"2429aa3e-046f-4ac9-b5d0-228476008ca6","Type":"ContainerDied","Data":"a79895615737d8bf1a47d4e36ac2f07cd0b0ff0261fded95be638f0481ee6f65"} Jul 18 01:57:15.152969 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:15.152947 2548 scope.go:117] "RemoveContainer" containerID="33a076e51fd53cf9c9e004c773575e2d4d7d132a5ca62d5d85c5ecc40235e016" Jul 18 01:57:15.161445 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:15.161429 2548 scope.go:117] "RemoveContainer" containerID="33a076e51fd53cf9c9e004c773575e2d4d7d132a5ca62d5d85c5ecc40235e016" Jul 18 01:57:15.161694 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:57:15.161677 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33a076e51fd53cf9c9e004c773575e2d4d7d132a5ca62d5d85c5ecc40235e016\": container with ID starting with 33a076e51fd53cf9c9e004c773575e2d4d7d132a5ca62d5d85c5ecc40235e016 not found: ID does not exist" containerID="33a076e51fd53cf9c9e004c773575e2d4d7d132a5ca62d5d85c5ecc40235e016" Jul 18 01:57:15.161752 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:15.161702 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33a076e51fd53cf9c9e004c773575e2d4d7d132a5ca62d5d85c5ecc40235e016"} err="failed to get container status \"33a076e51fd53cf9c9e004c773575e2d4d7d132a5ca62d5d85c5ecc40235e016\": rpc error: code = NotFound desc = could not find container \"33a076e51fd53cf9c9e004c773575e2d4d7d132a5ca62d5d85c5ecc40235e016\": container with ID starting with 33a076e51fd53cf9c9e004c773575e2d4d7d132a5ca62d5d85c5ecc40235e016 not found: ID does not exist" Jul 18 01:57:15.172138 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:15.172114 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ff5nd"] Jul 18 01:57:15.175735 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:15.175717 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ff5nd"] Jul 18 01:57:15.548849 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:15.548809 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2429aa3e-046f-4ac9-b5d0-228476008ca6" path="/var/lib/kubelet/pods/2429aa3e-046f-4ac9-b5d0-228476008ca6/volumes" Jul 18 01:57:16.642964 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.642932 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-nq7v6"] Jul 18 01:57:16.643416 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.643381 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2429aa3e-046f-4ac9-b5d0-228476008ca6" containerName="limitador" Jul 18 01:57:16.643416 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.643415 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="2429aa3e-046f-4ac9-b5d0-228476008ca6" containerName="limitador" Jul 18 01:57:16.643529 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.643432 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c47e4c7a-4040-4e47-b557-c2b12c3ef2dd" containerName="authorino" Jul 18 01:57:16.643529 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.643441 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="c47e4c7a-4040-4e47-b557-c2b12c3ef2dd" containerName="authorino" Jul 18 01:57:16.643529 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.643523 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="2429aa3e-046f-4ac9-b5d0-228476008ca6" containerName="limitador" Jul 18 01:57:16.643685 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.643533 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="c47e4c7a-4040-4e47-b557-c2b12c3ef2dd" containerName="authorino" Jul 18 01:57:16.647791 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.647753 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nq7v6" Jul 18 01:57:16.650008 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.649986 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 18 01:57:16.650148 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.650092 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-c7lpq\"" Jul 18 01:57:16.650238 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.650223 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 18 01:57:16.651035 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.651018 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 18 01:57:16.653770 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.653751 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-nq7v6"] Jul 18 01:57:16.692338 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.692311 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4txzm\" (UniqueName: \"kubernetes.io/projected/6019d639-5f08-4d25-a871-29197b9334dd-kube-api-access-4txzm\") pod \"postgres-69fb4594cf-nq7v6\" (UID: \"6019d639-5f08-4d25-a871-29197b9334dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nq7v6" Jul 18 01:57:16.692438 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.692382 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6019d639-5f08-4d25-a871-29197b9334dd-data\") pod \"postgres-69fb4594cf-nq7v6\" (UID: \"6019d639-5f08-4d25-a871-29197b9334dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nq7v6" Jul 18 01:57:16.792717 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.792693 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4txzm\" (UniqueName: \"kubernetes.io/projected/6019d639-5f08-4d25-a871-29197b9334dd-kube-api-access-4txzm\") pod \"postgres-69fb4594cf-nq7v6\" (UID: \"6019d639-5f08-4d25-a871-29197b9334dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nq7v6" Jul 18 01:57:16.792836 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.792754 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6019d639-5f08-4d25-a871-29197b9334dd-data\") pod \"postgres-69fb4594cf-nq7v6\" (UID: \"6019d639-5f08-4d25-a871-29197b9334dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nq7v6" Jul 18 01:57:16.793067 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.793050 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6019d639-5f08-4d25-a871-29197b9334dd-data\") pod \"postgres-69fb4594cf-nq7v6\" (UID: \"6019d639-5f08-4d25-a871-29197b9334dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nq7v6" Jul 18 01:57:16.801203 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.801181 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4txzm\" (UniqueName: \"kubernetes.io/projected/6019d639-5f08-4d25-a871-29197b9334dd-kube-api-access-4txzm\") pod \"postgres-69fb4594cf-nq7v6\" (UID: \"6019d639-5f08-4d25-a871-29197b9334dd\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nq7v6" Jul 18 01:57:16.959568 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:16.959542 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nq7v6" Jul 18 01:57:17.084740 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:17.084686 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-nq7v6"] Jul 18 01:57:17.088347 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:57:17.088308 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6019d639_5f08_4d25_a871_29197b9334dd.slice/crio-c0655b66acfda3b644dd0fecebf074fece704374fca5254756966f7af691cd8a WatchSource:0}: Error finding container c0655b66acfda3b644dd0fecebf074fece704374fca5254756966f7af691cd8a: Status 404 returned error can't find the container with id c0655b66acfda3b644dd0fecebf074fece704374fca5254756966f7af691cd8a Jul 18 01:57:17.166392 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:17.166367 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nq7v6" event={"ID":"6019d639-5f08-4d25-a871-29197b9334dd","Type":"ContainerStarted","Data":"c0655b66acfda3b644dd0fecebf074fece704374fca5254756966f7af691cd8a"} Jul 18 01:57:22.198653 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:22.198615 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nq7v6" event={"ID":"6019d639-5f08-4d25-a871-29197b9334dd","Type":"ContainerStarted","Data":"cfb3de18a7da7fe43047d65343bbc74c6d2fbbc4cd61dc1353273db2511adf5d"} Jul 18 01:57:22.199197 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:22.198796 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nq7v6" Jul 18 01:57:22.215835 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:22.215792 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nq7v6" podStartSLOduration=1.419430718 podStartE2EDuration="6.21576126s" totalImagesPullingTime="4.796330542s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:22.214474841 +0000 UTC m=+551.067218186" watchObservedRunningTime="2026-07-18 01:57:22.21576126 +0000 UTC m=+551.068504603" Jul 18 01:57:28.236338 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:28.236301 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nq7v6" Jul 18 01:57:28.780628 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:28.780597 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s26ql"] Jul 18 01:57:28.787364 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:28.787341 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-s26ql" Jul 18 01:57:28.789431 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:28.789408 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s26ql"] Jul 18 01:57:28.892119 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:28.892091 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw8kv\" (UniqueName: \"kubernetes.io/projected/ece51819-7732-467c-8718-98de2804672d-kube-api-access-mw8kv\") pod \"authorino-5d4b67c446-s26ql\" (UID: \"ece51819-7732-467c-8718-98de2804672d\") " pod="kuadrant-system/authorino-5d4b67c446-s26ql" Jul 18 01:57:28.993082 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:28.993051 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw8kv\" (UniqueName: \"kubernetes.io/projected/ece51819-7732-467c-8718-98de2804672d-kube-api-access-mw8kv\") pod \"authorino-5d4b67c446-s26ql\" (UID: \"ece51819-7732-467c-8718-98de2804672d\") " pod="kuadrant-system/authorino-5d4b67c446-s26ql" Jul 18 01:57:29.002519 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.002499 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw8kv\" (UniqueName: \"kubernetes.io/projected/ece51819-7732-467c-8718-98de2804672d-kube-api-access-mw8kv\") pod \"authorino-5d4b67c446-s26ql\" (UID: \"ece51819-7732-467c-8718-98de2804672d\") " pod="kuadrant-system/authorino-5d4b67c446-s26ql" Jul 18 01:57:29.022219 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.022195 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s26ql"] Jul 18 01:57:29.022388 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.022377 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-s26ql" Jul 18 01:57:29.048566 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.048543 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6659fc7f8b-7k6d5"] Jul 18 01:57:29.054349 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.054327 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6659fc7f8b-7k6d5" Jul 18 01:57:29.060627 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.060606 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6659fc7f8b-7k6d5"] Jul 18 01:57:29.094499 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.094462 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5h7x\" (UniqueName: \"kubernetes.io/projected/a0e9ca2f-0cdc-42b9-b295-32891bed2c30-kube-api-access-s5h7x\") pod \"authorino-6659fc7f8b-7k6d5\" (UID: \"a0e9ca2f-0cdc-42b9-b295-32891bed2c30\") " pod="kuadrant-system/authorino-6659fc7f8b-7k6d5" Jul 18 01:57:29.158568 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.158535 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6659fc7f8b-7k6d5"] Jul 18 01:57:29.158844 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:57:29.158824 2548 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-s5h7x], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6659fc7f8b-7k6d5" podUID="a0e9ca2f-0cdc-42b9-b295-32891bed2c30" Jul 18 01:57:29.165280 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.165256 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s26ql"] Jul 18 01:57:29.168727 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:57:29.168702 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podece51819_7732_467c_8718_98de2804672d.slice/crio-fd4e61e607526e1c66c172cf55a323265703fc21362c6a94b2c64c4d3e4823e9 WatchSource:0}: Error finding container fd4e61e607526e1c66c172cf55a323265703fc21362c6a94b2c64c4d3e4823e9: Status 404 returned error can't find the container with id fd4e61e607526e1c66c172cf55a323265703fc21362c6a94b2c64c4d3e4823e9 Jul 18 01:57:29.186645 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.186621 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bbbd47994-ts56j"] Jul 18 01:57:29.191065 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.191049 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bbbd47994-ts56j" Jul 18 01:57:29.193298 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.193276 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 18 01:57:29.195281 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.195264 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5h7x\" (UniqueName: \"kubernetes.io/projected/a0e9ca2f-0cdc-42b9-b295-32891bed2c30-kube-api-access-s5h7x\") pod \"authorino-6659fc7f8b-7k6d5\" (UID: \"a0e9ca2f-0cdc-42b9-b295-32891bed2c30\") " pod="kuadrant-system/authorino-6659fc7f8b-7k6d5" Jul 18 01:57:29.199215 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.199195 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bbbd47994-ts56j"] Jul 18 01:57:29.205691 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.205667 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5h7x\" (UniqueName: \"kubernetes.io/projected/a0e9ca2f-0cdc-42b9-b295-32891bed2c30-kube-api-access-s5h7x\") pod \"authorino-6659fc7f8b-7k6d5\" (UID: \"a0e9ca2f-0cdc-42b9-b295-32891bed2c30\") " pod="kuadrant-system/authorino-6659fc7f8b-7k6d5" Jul 18 01:57:29.241256 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.241232 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-s26ql" event={"ID":"ece51819-7732-467c-8718-98de2804672d","Type":"ContainerStarted","Data":"fd4e61e607526e1c66c172cf55a323265703fc21362c6a94b2c64c4d3e4823e9"} Jul 18 01:57:29.241559 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.241245 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6659fc7f8b-7k6d5" Jul 18 01:57:29.248561 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.248545 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6659fc7f8b-7k6d5" Jul 18 01:57:29.296581 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.296532 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5h7x\" (UniqueName: \"kubernetes.io/projected/a0e9ca2f-0cdc-42b9-b295-32891bed2c30-kube-api-access-s5h7x\") pod \"a0e9ca2f-0cdc-42b9-b295-32891bed2c30\" (UID: \"a0e9ca2f-0cdc-42b9-b295-32891bed2c30\") " Jul 18 01:57:29.296674 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.296659 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/53925eac-d4eb-49f4-9044-af939affe529-tls-cert\") pod \"authorino-7bbbd47994-ts56j\" (UID: \"53925eac-d4eb-49f4-9044-af939affe529\") " pod="kuadrant-system/authorino-7bbbd47994-ts56j" Jul 18 01:57:29.296723 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.296708 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb4t4\" (UniqueName: \"kubernetes.io/projected/53925eac-d4eb-49f4-9044-af939affe529-kube-api-access-jb4t4\") pod \"authorino-7bbbd47994-ts56j\" (UID: \"53925eac-d4eb-49f4-9044-af939affe529\") " pod="kuadrant-system/authorino-7bbbd47994-ts56j" Jul 18 01:57:29.298591 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.298570 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0e9ca2f-0cdc-42b9-b295-32891bed2c30-kube-api-access-s5h7x" (OuterVolumeSpecName: "kube-api-access-s5h7x") pod "a0e9ca2f-0cdc-42b9-b295-32891bed2c30" (UID: "a0e9ca2f-0cdc-42b9-b295-32891bed2c30"). InnerVolumeSpecName "kube-api-access-s5h7x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:57:29.397866 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.397846 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/53925eac-d4eb-49f4-9044-af939affe529-tls-cert\") pod \"authorino-7bbbd47994-ts56j\" (UID: \"53925eac-d4eb-49f4-9044-af939affe529\") " pod="kuadrant-system/authorino-7bbbd47994-ts56j" Jul 18 01:57:29.397973 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.397902 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb4t4\" (UniqueName: \"kubernetes.io/projected/53925eac-d4eb-49f4-9044-af939affe529-kube-api-access-jb4t4\") pod \"authorino-7bbbd47994-ts56j\" (UID: \"53925eac-d4eb-49f4-9044-af939affe529\") " pod="kuadrant-system/authorino-7bbbd47994-ts56j" Jul 18 01:57:29.397973 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.397952 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5h7x\" (UniqueName: \"kubernetes.io/projected/a0e9ca2f-0cdc-42b9-b295-32891bed2c30-kube-api-access-s5h7x\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:57:29.400151 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.400133 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/53925eac-d4eb-49f4-9044-af939affe529-tls-cert\") pod \"authorino-7bbbd47994-ts56j\" (UID: \"53925eac-d4eb-49f4-9044-af939affe529\") " pod="kuadrant-system/authorino-7bbbd47994-ts56j" Jul 18 01:57:29.405133 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.405114 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb4t4\" (UniqueName: \"kubernetes.io/projected/53925eac-d4eb-49f4-9044-af939affe529-kube-api-access-jb4t4\") pod \"authorino-7bbbd47994-ts56j\" (UID: \"53925eac-d4eb-49f4-9044-af939affe529\") " pod="kuadrant-system/authorino-7bbbd47994-ts56j" Jul 18 01:57:29.500603 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.500583 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bbbd47994-ts56j" Jul 18 01:57:29.670803 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:29.670756 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bbbd47994-ts56j"] Jul 18 01:57:29.672599 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:57:29.672571 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod53925eac_d4eb_49f4_9044_af939affe529.slice/crio-0d0fbcb30af636572f476efdb0758af6a236d5095e1cc3bf703e7762b36b4013 WatchSource:0}: Error finding container 0d0fbcb30af636572f476efdb0758af6a236d5095e1cc3bf703e7762b36b4013: Status 404 returned error can't find the container with id 0d0fbcb30af636572f476efdb0758af6a236d5095e1cc3bf703e7762b36b4013 Jul 18 01:57:30.248744 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.248639 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-s26ql" event={"ID":"ece51819-7732-467c-8718-98de2804672d","Type":"ContainerStarted","Data":"e13143a7720c734b9c914b559d7872d15ffbb1564bdcdbf5bf6d7b5806e1a523"} Jul 18 01:57:30.248744 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.248728 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-s26ql" podUID="ece51819-7732-467c-8718-98de2804672d" containerName="authorino" containerID="cri-o://e13143a7720c734b9c914b559d7872d15ffbb1564bdcdbf5bf6d7b5806e1a523" gracePeriod=30 Jul 18 01:57:30.250541 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.250503 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bbbd47994-ts56j" event={"ID":"53925eac-d4eb-49f4-9044-af939affe529","Type":"ContainerStarted","Data":"0e217f043a7fa3ab6e51de746e3d7884344fa45afb2a6a40198605b6306ed4f8"} Jul 18 01:57:30.250670 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.250548 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bbbd47994-ts56j" event={"ID":"53925eac-d4eb-49f4-9044-af939affe529","Type":"ContainerStarted","Data":"0d0fbcb30af636572f476efdb0758af6a236d5095e1cc3bf703e7762b36b4013"} Jul 18 01:57:30.250670 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.250574 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6659fc7f8b-7k6d5" Jul 18 01:57:30.264114 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.264033 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-s26ql" podStartSLOduration=1.9702876580000002 podStartE2EDuration="2.264018327s" totalImagesPullingTime="293.730669ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:30.262892026 +0000 UTC m=+559.115635383" watchObservedRunningTime="2026-07-18 01:57:30.264018327 +0000 UTC m=+559.116761671" Jul 18 01:57:30.277237 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.277198 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bbbd47994-ts56j" podStartSLOduration=0.818002145 podStartE2EDuration="1.277184894s" totalImagesPullingTime="459.182749ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:30.275704863 +0000 UTC m=+559.128448208" watchObservedRunningTime="2026-07-18 01:57:30.277184894 +0000 UTC m=+559.129928237" Jul 18 01:57:30.297533 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.297506 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6659fc7f8b-7k6d5"] Jul 18 01:57:30.304227 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.304192 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6659fc7f8b-7k6d5"] Jul 18 01:57:30.305647 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.305624 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-72s9r"] Jul 18 01:57:30.305814 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.305789 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-72s9r" podUID="07a70a9a-da9c-4e13-b7c0-71041c2d56a4" containerName="authorino" containerID="cri-o://e1f948933f4368e89fcbecd342670a40bee17487839d6131d1c471fe2718d6c8" gracePeriod=30 Jul 18 01:57:30.536800 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.536757 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-s26ql" Jul 18 01:57:30.570847 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.570824 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-72s9r" Jul 18 01:57:30.607645 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.607614 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mw8kv\" (UniqueName: \"kubernetes.io/projected/ece51819-7732-467c-8718-98de2804672d-kube-api-access-mw8kv\") pod \"ece51819-7732-467c-8718-98de2804672d\" (UID: \"ece51819-7732-467c-8718-98de2804672d\") " Jul 18 01:57:30.607768 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.607672 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkf4k\" (UniqueName: \"kubernetes.io/projected/07a70a9a-da9c-4e13-b7c0-71041c2d56a4-kube-api-access-zkf4k\") pod \"07a70a9a-da9c-4e13-b7c0-71041c2d56a4\" (UID: \"07a70a9a-da9c-4e13-b7c0-71041c2d56a4\") " Jul 18 01:57:30.609559 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.609531 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ece51819-7732-467c-8718-98de2804672d-kube-api-access-mw8kv" (OuterVolumeSpecName: "kube-api-access-mw8kv") pod "ece51819-7732-467c-8718-98de2804672d" (UID: "ece51819-7732-467c-8718-98de2804672d"). InnerVolumeSpecName "kube-api-access-mw8kv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:57:30.609659 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.609560 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07a70a9a-da9c-4e13-b7c0-71041c2d56a4-kube-api-access-zkf4k" (OuterVolumeSpecName: "kube-api-access-zkf4k") pod "07a70a9a-da9c-4e13-b7c0-71041c2d56a4" (UID: "07a70a9a-da9c-4e13-b7c0-71041c2d56a4"). InnerVolumeSpecName "kube-api-access-zkf4k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 01:57:30.708754 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.708729 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mw8kv\" (UniqueName: \"kubernetes.io/projected/ece51819-7732-467c-8718-98de2804672d-kube-api-access-mw8kv\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:57:30.708754 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:30.708751 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkf4k\" (UniqueName: \"kubernetes.io/projected/07a70a9a-da9c-4e13-b7c0-71041c2d56a4-kube-api-access-zkf4k\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 01:57:31.256595 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.256554 2548 generic.go:356] "Generic (PLEG): container finished" podID="07a70a9a-da9c-4e13-b7c0-71041c2d56a4" containerID="e1f948933f4368e89fcbecd342670a40bee17487839d6131d1c471fe2718d6c8" exitCode=0 Jul 18 01:57:31.257082 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.256603 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-72s9r" Jul 18 01:57:31.257082 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.256648 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-72s9r" event={"ID":"07a70a9a-da9c-4e13-b7c0-71041c2d56a4","Type":"ContainerDied","Data":"e1f948933f4368e89fcbecd342670a40bee17487839d6131d1c471fe2718d6c8"} Jul 18 01:57:31.257082 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.256679 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-72s9r" event={"ID":"07a70a9a-da9c-4e13-b7c0-71041c2d56a4","Type":"ContainerDied","Data":"87699cdcee4ed2c2c711e6549b55aa6de629d3bff0710a41aeb12a0e69a9278b"} Jul 18 01:57:31.257082 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.256699 2548 scope.go:117] "RemoveContainer" containerID="e1f948933f4368e89fcbecd342670a40bee17487839d6131d1c471fe2718d6c8" Jul 18 01:57:31.258138 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.258069 2548 generic.go:356] "Generic (PLEG): container finished" podID="ece51819-7732-467c-8718-98de2804672d" containerID="e13143a7720c734b9c914b559d7872d15ffbb1564bdcdbf5bf6d7b5806e1a523" exitCode=0 Jul 18 01:57:31.258138 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.258117 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-s26ql" Jul 18 01:57:31.258277 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.258162 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-s26ql" event={"ID":"ece51819-7732-467c-8718-98de2804672d","Type":"ContainerDied","Data":"e13143a7720c734b9c914b559d7872d15ffbb1564bdcdbf5bf6d7b5806e1a523"} Jul 18 01:57:31.258277 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.258204 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-s26ql" event={"ID":"ece51819-7732-467c-8718-98de2804672d","Type":"ContainerDied","Data":"fd4e61e607526e1c66c172cf55a323265703fc21362c6a94b2c64c4d3e4823e9"} Jul 18 01:57:31.266423 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.266406 2548 scope.go:117] "RemoveContainer" containerID="e1f948933f4368e89fcbecd342670a40bee17487839d6131d1c471fe2718d6c8" Jul 18 01:57:31.266662 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:57:31.266646 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1f948933f4368e89fcbecd342670a40bee17487839d6131d1c471fe2718d6c8\": container with ID starting with e1f948933f4368e89fcbecd342670a40bee17487839d6131d1c471fe2718d6c8 not found: ID does not exist" containerID="e1f948933f4368e89fcbecd342670a40bee17487839d6131d1c471fe2718d6c8" Jul 18 01:57:31.266719 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.266670 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1f948933f4368e89fcbecd342670a40bee17487839d6131d1c471fe2718d6c8"} err="failed to get container status \"e1f948933f4368e89fcbecd342670a40bee17487839d6131d1c471fe2718d6c8\": rpc error: code = NotFound desc = could not find container \"e1f948933f4368e89fcbecd342670a40bee17487839d6131d1c471fe2718d6c8\": container with ID starting with e1f948933f4368e89fcbecd342670a40bee17487839d6131d1c471fe2718d6c8 not found: ID does not exist" Jul 18 01:57:31.266719 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.266688 2548 scope.go:117] "RemoveContainer" containerID="e13143a7720c734b9c914b559d7872d15ffbb1564bdcdbf5bf6d7b5806e1a523" Jul 18 01:57:31.274181 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.274165 2548 scope.go:117] "RemoveContainer" containerID="e13143a7720c734b9c914b559d7872d15ffbb1564bdcdbf5bf6d7b5806e1a523" Jul 18 01:57:31.274408 ip-10-0-137-108 kubenswrapper[2548]: E0718 01:57:31.274392 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e13143a7720c734b9c914b559d7872d15ffbb1564bdcdbf5bf6d7b5806e1a523\": container with ID starting with e13143a7720c734b9c914b559d7872d15ffbb1564bdcdbf5bf6d7b5806e1a523 not found: ID does not exist" containerID="e13143a7720c734b9c914b559d7872d15ffbb1564bdcdbf5bf6d7b5806e1a523" Jul 18 01:57:31.274462 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.274413 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e13143a7720c734b9c914b559d7872d15ffbb1564bdcdbf5bf6d7b5806e1a523"} err="failed to get container status \"e13143a7720c734b9c914b559d7872d15ffbb1564bdcdbf5bf6d7b5806e1a523\": rpc error: code = NotFound desc = could not find container \"e13143a7720c734b9c914b559d7872d15ffbb1564bdcdbf5bf6d7b5806e1a523\": container with ID starting with e13143a7720c734b9c914b559d7872d15ffbb1564bdcdbf5bf6d7b5806e1a523 not found: ID does not exist" Jul 18 01:57:31.296252 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.296226 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-72s9r"] Jul 18 01:57:31.307622 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.307601 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-72s9r"] Jul 18 01:57:31.350956 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.350934 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s26ql"] Jul 18 01:57:31.358424 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.358401 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-s26ql"] Jul 18 01:57:31.548740 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.548655 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07a70a9a-da9c-4e13-b7c0-71041c2d56a4" path="/var/lib/kubelet/pods/07a70a9a-da9c-4e13-b7c0-71041c2d56a4/volumes" Jul 18 01:57:31.549056 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.549037 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0e9ca2f-0cdc-42b9-b295-32891bed2c30" path="/var/lib/kubelet/pods/a0e9ca2f-0cdc-42b9-b295-32891bed2c30/volumes" Jul 18 01:57:31.549281 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:31.549264 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ece51819-7732-467c-8718-98de2804672d" path="/var/lib/kubelet/pods/ece51819-7732-467c-8718-98de2804672d/volumes" Jul 18 01:57:35.079626 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.079596 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75667cc8f5-tswqq"] Jul 18 01:57:35.080012 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.079950 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07a70a9a-da9c-4e13-b7c0-71041c2d56a4" containerName="authorino" Jul 18 01:57:35.080012 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.079961 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="07a70a9a-da9c-4e13-b7c0-71041c2d56a4" containerName="authorino" Jul 18 01:57:35.080012 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.079977 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ece51819-7732-467c-8718-98de2804672d" containerName="authorino" Jul 18 01:57:35.080012 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.079983 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="ece51819-7732-467c-8718-98de2804672d" containerName="authorino" Jul 18 01:57:35.080132 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.080065 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="07a70a9a-da9c-4e13-b7c0-71041c2d56a4" containerName="authorino" Jul 18 01:57:35.080132 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.080074 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="ece51819-7732-467c-8718-98de2804672d" containerName="authorino" Jul 18 01:57:35.090101 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.090072 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75667cc8f5-tswqq" Jul 18 01:57:35.090378 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.090351 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75667cc8f5-tswqq"] Jul 18 01:57:35.092419 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.092393 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 18 01:57:35.092578 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.092555 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zpvtf\"" Jul 18 01:57:35.092724 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.092657 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 18 01:57:35.150995 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.150964 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkfdw\" (UniqueName: \"kubernetes.io/projected/42eef91e-9264-460e-b983-f5c61e6d5f27-kube-api-access-rkfdw\") pod \"maas-controller-75667cc8f5-tswqq\" (UID: \"42eef91e-9264-460e-b983-f5c61e6d5f27\") " pod="opendatahub/maas-controller-75667cc8f5-tswqq" Jul 18 01:57:35.151104 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.151043 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42eef91e-9264-460e-b983-f5c61e6d5f27-webhook-cert\") pod \"maas-controller-75667cc8f5-tswqq\" (UID: \"42eef91e-9264-460e-b983-f5c61e6d5f27\") " pod="opendatahub/maas-controller-75667cc8f5-tswqq" Jul 18 01:57:35.252416 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.252386 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkfdw\" (UniqueName: \"kubernetes.io/projected/42eef91e-9264-460e-b983-f5c61e6d5f27-kube-api-access-rkfdw\") pod \"maas-controller-75667cc8f5-tswqq\" (UID: \"42eef91e-9264-460e-b983-f5c61e6d5f27\") " pod="opendatahub/maas-controller-75667cc8f5-tswqq" Jul 18 01:57:35.252564 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.252497 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42eef91e-9264-460e-b983-f5c61e6d5f27-webhook-cert\") pod \"maas-controller-75667cc8f5-tswqq\" (UID: \"42eef91e-9264-460e-b983-f5c61e6d5f27\") " pod="opendatahub/maas-controller-75667cc8f5-tswqq" Jul 18 01:57:35.254921 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.254900 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42eef91e-9264-460e-b983-f5c61e6d5f27-webhook-cert\") pod \"maas-controller-75667cc8f5-tswqq\" (UID: \"42eef91e-9264-460e-b983-f5c61e6d5f27\") " pod="opendatahub/maas-controller-75667cc8f5-tswqq" Jul 18 01:57:35.260308 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.260287 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkfdw\" (UniqueName: \"kubernetes.io/projected/42eef91e-9264-460e-b983-f5c61e6d5f27-kube-api-access-rkfdw\") pod \"maas-controller-75667cc8f5-tswqq\" (UID: \"42eef91e-9264-460e-b983-f5c61e6d5f27\") " pod="opendatahub/maas-controller-75667cc8f5-tswqq" Jul 18 01:57:35.402022 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.401997 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75667cc8f5-tswqq" Jul 18 01:57:35.540744 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:35.540719 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75667cc8f5-tswqq"] Jul 18 01:57:35.542883 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:57:35.542848 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42eef91e_9264_460e_b983_f5c61e6d5f27.slice/crio-f46beb7527c94c3fefe22dea8fb0d5bc1174eb5a0f1d944e18cb52a9377031fb WatchSource:0}: Error finding container f46beb7527c94c3fefe22dea8fb0d5bc1174eb5a0f1d944e18cb52a9377031fb: Status 404 returned error can't find the container with id f46beb7527c94c3fefe22dea8fb0d5bc1174eb5a0f1d944e18cb52a9377031fb Jul 18 01:57:36.289501 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:36.289464 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75667cc8f5-tswqq" event={"ID":"42eef91e-9264-460e-b983-f5c61e6d5f27","Type":"ContainerStarted","Data":"f46beb7527c94c3fefe22dea8fb0d5bc1174eb5a0f1d944e18cb52a9377031fb"} Jul 18 01:57:39.308218 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:39.308181 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75667cc8f5-tswqq" event={"ID":"42eef91e-9264-460e-b983-f5c61e6d5f27","Type":"ContainerStarted","Data":"363eb8d5a782937fc0d54defe1c48b46e053f57be1f1119dbe9d4c35ae7d49da"} Jul 18 01:57:39.308657 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:39.308270 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75667cc8f5-tswqq" Jul 18 01:57:39.332456 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:39.332418 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75667cc8f5-tswqq" podStartSLOduration=1.3805140040000001 podStartE2EDuration="4.332407203s" totalImagesPullingTime="2.951893199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:39.332366709 +0000 UTC m=+568.185110054" watchObservedRunningTime="2026-07-18 01:57:39.332407203 +0000 UTC m=+568.185150547" Jul 18 01:57:50.316331 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:57:50.316303 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75667cc8f5-tswqq" Jul 18 01:58:10.891921 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:10.891891 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl"] Jul 18 01:58:10.896226 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:10.896209 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl" Jul 18 01:58:10.898860 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:10.898824 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 18 01:58:10.899001 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:10.898828 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-gnmnn\"" Jul 18 01:58:10.903714 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:10.903692 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl"] Jul 18 01:58:11.042266 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.042232 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khnzh\" (UniqueName: \"kubernetes.io/projected/edb6f98f-114b-4bcf-a091-a975c6bcdc39-kube-api-access-khnzh\") pod \"maas-api-6576d4df98-sqdwl\" (UID: \"edb6f98f-114b-4bcf-a091-a975c6bcdc39\") " pod="odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl" Jul 18 01:58:11.042413 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.042288 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edb6f98f-114b-4bcf-a091-a975c6bcdc39-maas-api-tls\") pod \"maas-api-6576d4df98-sqdwl\" (UID: \"edb6f98f-114b-4bcf-a091-a975c6bcdc39\") " pod="odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl" Jul 18 01:58:11.042413 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.042368 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edb6f98f-114b-4bcf-a091-a975c6bcdc39-openshift-service-ca\") pod \"maas-api-6576d4df98-sqdwl\" (UID: \"edb6f98f-114b-4bcf-a091-a975c6bcdc39\") " pod="odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl" Jul 18 01:58:11.143614 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.143541 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khnzh\" (UniqueName: \"kubernetes.io/projected/edb6f98f-114b-4bcf-a091-a975c6bcdc39-kube-api-access-khnzh\") pod \"maas-api-6576d4df98-sqdwl\" (UID: \"edb6f98f-114b-4bcf-a091-a975c6bcdc39\") " pod="odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl" Jul 18 01:58:11.143614 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.143595 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edb6f98f-114b-4bcf-a091-a975c6bcdc39-maas-api-tls\") pod \"maas-api-6576d4df98-sqdwl\" (UID: \"edb6f98f-114b-4bcf-a091-a975c6bcdc39\") " pod="odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl" Jul 18 01:58:11.143852 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.143655 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edb6f98f-114b-4bcf-a091-a975c6bcdc39-openshift-service-ca\") pod \"maas-api-6576d4df98-sqdwl\" (UID: \"edb6f98f-114b-4bcf-a091-a975c6bcdc39\") " pod="odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl" Jul 18 01:58:11.144292 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.144269 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/edb6f98f-114b-4bcf-a091-a975c6bcdc39-openshift-service-ca\") pod \"maas-api-6576d4df98-sqdwl\" (UID: \"edb6f98f-114b-4bcf-a091-a975c6bcdc39\") " pod="odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl" Jul 18 01:58:11.145957 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.145932 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/edb6f98f-114b-4bcf-a091-a975c6bcdc39-maas-api-tls\") pod \"maas-api-6576d4df98-sqdwl\" (UID: \"edb6f98f-114b-4bcf-a091-a975c6bcdc39\") " pod="odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl" Jul 18 01:58:11.154351 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.154326 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khnzh\" (UniqueName: \"kubernetes.io/projected/edb6f98f-114b-4bcf-a091-a975c6bcdc39-kube-api-access-khnzh\") pod \"maas-api-6576d4df98-sqdwl\" (UID: \"edb6f98f-114b-4bcf-a091-a975c6bcdc39\") " pod="odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl" Jul 18 01:58:11.209147 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.209116 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl" Jul 18 01:58:11.360584 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.360557 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl"] Jul 18 01:58:11.363435 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:58:11.363410 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedb6f98f_114b_4bcf_a091_a975c6bcdc39.slice/crio-413e98f2fbb28d4275b02fa1ea1ea95f3202de61e96e6a44da301508578e9bb8 WatchSource:0}: Error finding container 413e98f2fbb28d4275b02fa1ea1ea95f3202de61e96e6a44da301508578e9bb8: Status 404 returned error can't find the container with id 413e98f2fbb28d4275b02fa1ea1ea95f3202de61e96e6a44da301508578e9bb8 Jul 18 01:58:11.433290 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.433265 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-v8tzt"] Jul 18 01:58:11.472152 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.472126 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-v8tzt"] Jul 18 01:58:11.472327 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.472161 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl" event={"ID":"edb6f98f-114b-4bcf-a091-a975c6bcdc39","Type":"ContainerStarted","Data":"413e98f2fbb28d4275b02fa1ea1ea95f3202de61e96e6a44da301508578e9bb8"} Jul 18 01:58:11.472327 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.472255 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-v8tzt" Jul 18 01:58:11.474458 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.474432 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 18 01:58:11.474575 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.474465 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-n757b\"" Jul 18 01:58:11.482997 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.482977 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 01:58:11.484218 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.484201 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 01:58:11.649865 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.649834 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da9d422-36df-404f-a158-d3de9a1b6a6a-plugins-config-volume\") pod \"payload-processing-5bf47cd988-v8tzt\" (UID: \"6da9d422-36df-404f-a158-d3de9a1b6a6a\") " pod="openshift-ingress/payload-processing-5bf47cd988-v8tzt" Jul 18 01:58:11.650061 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.649877 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rflnj\" (UniqueName: \"kubernetes.io/projected/6da9d422-36df-404f-a158-d3de9a1b6a6a-kube-api-access-rflnj\") pod \"payload-processing-5bf47cd988-v8tzt\" (UID: \"6da9d422-36df-404f-a158-d3de9a1b6a6a\") " pod="openshift-ingress/payload-processing-5bf47cd988-v8tzt" Jul 18 01:58:11.750616 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.750529 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da9d422-36df-404f-a158-d3de9a1b6a6a-plugins-config-volume\") pod \"payload-processing-5bf47cd988-v8tzt\" (UID: \"6da9d422-36df-404f-a158-d3de9a1b6a6a\") " pod="openshift-ingress/payload-processing-5bf47cd988-v8tzt" Jul 18 01:58:11.750616 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.750571 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rflnj\" (UniqueName: \"kubernetes.io/projected/6da9d422-36df-404f-a158-d3de9a1b6a6a-kube-api-access-rflnj\") pod \"payload-processing-5bf47cd988-v8tzt\" (UID: \"6da9d422-36df-404f-a158-d3de9a1b6a6a\") " pod="openshift-ingress/payload-processing-5bf47cd988-v8tzt" Jul 18 01:58:11.751301 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.751275 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6da9d422-36df-404f-a158-d3de9a1b6a6a-plugins-config-volume\") pod \"payload-processing-5bf47cd988-v8tzt\" (UID: \"6da9d422-36df-404f-a158-d3de9a1b6a6a\") " pod="openshift-ingress/payload-processing-5bf47cd988-v8tzt" Jul 18 01:58:11.758308 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.758289 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rflnj\" (UniqueName: \"kubernetes.io/projected/6da9d422-36df-404f-a158-d3de9a1b6a6a-kube-api-access-rflnj\") pod \"payload-processing-5bf47cd988-v8tzt\" (UID: \"6da9d422-36df-404f-a158-d3de9a1b6a6a\") " pod="openshift-ingress/payload-processing-5bf47cd988-v8tzt" Jul 18 01:58:11.785759 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.785737 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-v8tzt" Jul 18 01:58:11.920401 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:11.920371 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-v8tzt"] Jul 18 01:58:11.922892 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:58:11.922855 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6da9d422_36df_404f_a158_d3de9a1b6a6a.slice/crio-6b4241b097da7347b0489236bf6693827bb4db283d43cbfd561ae26a4323b6b3 WatchSource:0}: Error finding container 6b4241b097da7347b0489236bf6693827bb4db283d43cbfd561ae26a4323b6b3: Status 404 returned error can't find the container with id 6b4241b097da7347b0489236bf6693827bb4db283d43cbfd561ae26a4323b6b3 Jul 18 01:58:12.477603 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:12.477559 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-v8tzt" event={"ID":"6da9d422-36df-404f-a158-d3de9a1b6a6a","Type":"ContainerStarted","Data":"6b4241b097da7347b0489236bf6693827bb4db283d43cbfd561ae26a4323b6b3"} Jul 18 01:58:13.484323 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:13.484279 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl" event={"ID":"edb6f98f-114b-4bcf-a091-a975c6bcdc39","Type":"ContainerStarted","Data":"8b3b2f47940bad004ccb348cc7e2c73c2f13a2a2015ee7b998bec43912e0814c"} Jul 18 01:58:13.484663 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:13.484419 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl" Jul 18 01:58:13.501840 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:13.501725 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl" podStartSLOduration=1.490681747 podStartE2EDuration="3.501709138s" totalImagesPullingTime="2.011027391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:13.501415709 +0000 UTC m=+602.354159050" watchObservedRunningTime="2026-07-18 01:58:13.501709138 +0000 UTC m=+602.354452482" Jul 18 01:58:15.495634 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:15.495588 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-v8tzt" event={"ID":"6da9d422-36df-404f-a158-d3de9a1b6a6a","Type":"ContainerStarted","Data":"16c6cc15893f0fbedecae077d2762b20038b4a615a567be0c43523c1f9e43ef7"} Jul 18 01:58:15.496038 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:15.495692 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-v8tzt" Jul 18 01:58:15.511875 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:15.511823 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-v8tzt" podStartSLOduration=1.631470215 podStartE2EDuration="4.511809986s" totalImagesPullingTime="2.880339771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:15.509964212 +0000 UTC m=+604.362707577" watchObservedRunningTime="2026-07-18 01:58:15.511809986 +0000 UTC m=+604.364553329" Jul 18 01:58:19.495005 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:19.494979 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6576d4df98-sqdwl" Jul 18 01:58:19.516464 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:19.516437 2548 generic.go:356] "Generic (PLEG): container finished" podID="aeddc1c9-3b8b-4f5b-91a8-ac2655950649" containerID="39784ed588e18351ae2ca964e020bcb3485f37b0b4f7d1d88dcfca0d2e2b8530" exitCode=0 Jul 18 01:58:19.516597 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:19.516502 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" event={"ID":"aeddc1c9-3b8b-4f5b-91a8-ac2655950649","Type":"ContainerDied","Data":"39784ed588e18351ae2ca964e020bcb3485f37b0b4f7d1d88dcfca0d2e2b8530"} Jul 18 01:58:19.516844 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:19.516827 2548 scope.go:117] "RemoveContainer" containerID="39784ed588e18351ae2ca964e020bcb3485f37b0b4f7d1d88dcfca0d2e2b8530" Jul 18 01:58:19.517887 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:19.517798 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:58:19.518009 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:19.517884 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:58:19.518009 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:19.517936 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 01:58:20.529489 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:20.529450 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" event={"ID":"aeddc1c9-3b8b-4f5b-91a8-ac2655950649","Type":"ContainerStarted","Data":"e0ffeb6ef3733b122188a973303a8f138b86e11d3e54b5e05e216fce23763b2f"} Jul 18 01:58:20.646688 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:20.646657 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:58:20.647020 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:20.646997 2548 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-62vpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 18 01:58:20.647162 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:20.647139 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" podUID="aeddc1c9-3b8b-4f5b-91a8-ac2655950649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 18 01:58:21.667656 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:21.667620 2548 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-62vpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 18 01:58:21.668273 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:21.668243 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" podUID="aeddc1c9-3b8b-4f5b-91a8-ac2655950649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 18 01:58:22.646727 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:22.646691 2548 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-62vpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 18 01:58:22.647034 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:22.647006 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" podUID="aeddc1c9-3b8b-4f5b-91a8-ac2655950649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 18 01:58:23.646975 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:23.646942 2548 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-62vpz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 18 01:58:23.647524 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:23.647492 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" podUID="aeddc1c9-3b8b-4f5b-91a8-ac2655950649" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 18 01:58:24.649230 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:24.649202 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:58:24.649632 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:24.649468 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:58:24.650145 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:24.650125 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-62vpz" Jul 18 01:58:26.502269 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:26.502243 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-v8tzt" Jul 18 01:58:28.305972 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.305934 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6"] Jul 18 01:58:28.309945 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.309922 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.313175 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.313147 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mb98f\"" Jul 18 01:58:28.313297 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.313153 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 18 01:58:28.313297 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.313155 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 18 01:58:28.313297 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.313154 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 18 01:58:28.319190 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.319166 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6"] Jul 18 01:58:28.418741 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.418709 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3008b30-7e0b-455e-a326-e322ec934289-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.418741 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.418740 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3008b30-7e0b-455e-a326-e322ec934289-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.418984 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.418760 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3008b30-7e0b-455e-a326-e322ec934289-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.418984 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.418928 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3008b30-7e0b-455e-a326-e322ec934289-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.418984 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.418965 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb9d6\" (UniqueName: \"kubernetes.io/projected/e3008b30-7e0b-455e-a326-e322ec934289-kube-api-access-jb9d6\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.419123 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.419053 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3008b30-7e0b-455e-a326-e322ec934289-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.519978 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.519948 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3008b30-7e0b-455e-a326-e322ec934289-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.520134 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.520007 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3008b30-7e0b-455e-a326-e322ec934289-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.520134 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.520034 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3008b30-7e0b-455e-a326-e322ec934289-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.520134 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.520058 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3008b30-7e0b-455e-a326-e322ec934289-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.520134 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.520132 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3008b30-7e0b-455e-a326-e322ec934289-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.520331 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.520159 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jb9d6\" (UniqueName: \"kubernetes.io/projected/e3008b30-7e0b-455e-a326-e322ec934289-kube-api-access-jb9d6\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.520473 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.520446 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3008b30-7e0b-455e-a326-e322ec934289-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.520607 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.520572 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3008b30-7e0b-455e-a326-e322ec934289-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.520676 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.520609 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3008b30-7e0b-455e-a326-e322ec934289-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.522316 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.522292 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3008b30-7e0b-455e-a326-e322ec934289-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.522564 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.522538 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3008b30-7e0b-455e-a326-e322ec934289-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.527990 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.527965 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb9d6\" (UniqueName: \"kubernetes.io/projected/e3008b30-7e0b-455e-a326-e322ec934289-kube-api-access-jb9d6\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6\" (UID: \"e3008b30-7e0b-455e-a326-e322ec934289\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.621388 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.621321 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:28.753944 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:28.753814 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6"] Jul 18 01:58:28.756381 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:58:28.756349 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3008b30_7e0b_455e_a326_e322ec934289.slice/crio-1b7255fb138e95bf70b38e65138b25fb559f4146641b6ce0b2166ec5acc152e9 WatchSource:0}: Error finding container 1b7255fb138e95bf70b38e65138b25fb559f4146641b6ce0b2166ec5acc152e9: Status 404 returned error can't find the container with id 1b7255fb138e95bf70b38e65138b25fb559f4146641b6ce0b2166ec5acc152e9 Jul 18 01:58:29.576623 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:29.576589 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" event={"ID":"e3008b30-7e0b-455e-a326-e322ec934289","Type":"ContainerStarted","Data":"1b7255fb138e95bf70b38e65138b25fb559f4146641b6ce0b2166ec5acc152e9"} Jul 18 01:58:30.582660 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:30.582627 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" event={"ID":"e3008b30-7e0b-455e-a326-e322ec934289","Type":"ContainerStarted","Data":"50bf1be837840d7db83b74afb4ea61bffa970215a86cced6bacd632eb6cfabd7"} Jul 18 01:58:30.601963 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:30.601913 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" podStartSLOduration=1.501899034 podStartE2EDuration="2.601899202s" totalImagesPullingTime="1.100000168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:30.599484511 +0000 UTC m=+619.452227855" watchObservedRunningTime="2026-07-18 01:58:30.601899202 +0000 UTC m=+619.454642546" Jul 18 01:58:38.621733 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:38.621658 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:38.621733 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:38.621705 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:38.633746 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:38.633725 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:39.635563 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:39.635528 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6" Jul 18 01:58:49.699176 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.699143 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc"] Jul 18 01:58:49.705335 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.705309 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.707749 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.707727 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 18 01:58:49.714665 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.714642 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc"] Jul 18 01:58:49.806219 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.806194 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.806372 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.806230 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.806372 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.806266 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.806372 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.806281 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.806372 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.806305 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.806372 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.806326 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bscg\" (UniqueName: \"kubernetes.io/projected/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-kube-api-access-6bscg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.907547 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.907516 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.907712 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.907558 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.907712 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.907589 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.907712 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.907615 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bscg\" (UniqueName: \"kubernetes.io/projected/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-kube-api-access-6bscg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.907712 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.907679 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.907979 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.907724 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.907979 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.907947 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.908094 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.908015 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.908149 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.908116 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.909741 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.909719 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.909950 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.909933 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:49.915745 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:49.915726 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bscg\" (UniqueName: \"kubernetes.io/projected/e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973-kube-api-access-6bscg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-mqssc\" (UID: \"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:50.016293 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:50.016224 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:58:50.157373 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:50.157348 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc"] Jul 18 01:58:50.159544 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:58:50.159503 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode82e0f44_cb4e_4d1c_bc8a_e5f2d9d48973.slice/crio-2e51ef37bc7f32f042c621c4ef55fbfb169200195be33fc3864731f94aef29d8 WatchSource:0}: Error finding container 2e51ef37bc7f32f042c621c4ef55fbfb169200195be33fc3864731f94aef29d8: Status 404 returned error can't find the container with id 2e51ef37bc7f32f042c621c4ef55fbfb169200195be33fc3864731f94aef29d8 Jul 18 01:58:50.686161 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:50.686125 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" event={"ID":"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973","Type":"ContainerStarted","Data":"11fe51273f5bb386ba2b03830a5ff80d94896cb36b1e53331e8c8325a142eaca"} Jul 18 01:58:50.686354 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:50.686181 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" event={"ID":"e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973","Type":"ContainerStarted","Data":"2e51ef37bc7f32f042c621c4ef55fbfb169200195be33fc3864731f94aef29d8"} Jul 18 01:58:50.702373 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:58:50.702326 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" podStartSLOduration=1.538665958 podStartE2EDuration="1.702310468s" totalImagesPullingTime="163.64451ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:50.701087299 +0000 UTC m=+639.553830655" watchObservedRunningTime="2026-07-18 01:58:50.702310468 +0000 UTC m=+639.555053811" Jul 18 01:59:00.017106 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:00.017077 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:59:00.017475 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:00.017147 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:59:00.029757 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:00.029736 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:59:00.746639 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:00.746610 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-mqssc" Jul 18 01:59:16.497735 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.497698 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x"] Jul 18 01:59:16.501579 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.501560 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.504001 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.503979 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 18 01:59:16.510296 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.510275 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x"] Jul 18 01:59:16.531448 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.531424 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acfe2150-68c0-4f24-bb4d-73a658ea6068-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.531571 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.531469 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttzwb\" (UniqueName: \"kubernetes.io/projected/acfe2150-68c0-4f24-bb4d-73a658ea6068-kube-api-access-ttzwb\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.531571 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.531488 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acfe2150-68c0-4f24-bb4d-73a658ea6068-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.531571 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.531529 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acfe2150-68c0-4f24-bb4d-73a658ea6068-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.531571 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.531563 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acfe2150-68c0-4f24-bb4d-73a658ea6068-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.531716 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.531609 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acfe2150-68c0-4f24-bb4d-73a658ea6068-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.632630 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.632599 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acfe2150-68c0-4f24-bb4d-73a658ea6068-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.632748 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.632694 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttzwb\" (UniqueName: \"kubernetes.io/projected/acfe2150-68c0-4f24-bb4d-73a658ea6068-kube-api-access-ttzwb\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.632748 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.632739 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acfe2150-68c0-4f24-bb4d-73a658ea6068-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.632902 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.632845 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acfe2150-68c0-4f24-bb4d-73a658ea6068-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.632902 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.632883 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acfe2150-68c0-4f24-bb4d-73a658ea6068-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.633016 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.632961 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acfe2150-68c0-4f24-bb4d-73a658ea6068-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.633162 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.633134 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acfe2150-68c0-4f24-bb4d-73a658ea6068-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.633340 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.633312 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acfe2150-68c0-4f24-bb4d-73a658ea6068-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.633444 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.633326 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acfe2150-68c0-4f24-bb4d-73a658ea6068-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.635497 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.635475 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acfe2150-68c0-4f24-bb4d-73a658ea6068-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.635497 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.635491 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acfe2150-68c0-4f24-bb4d-73a658ea6068-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.640792 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.640759 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttzwb\" (UniqueName: \"kubernetes.io/projected/acfe2150-68c0-4f24-bb4d-73a658ea6068-kube-api-access-ttzwb\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x\" (UID: \"acfe2150-68c0-4f24-bb4d-73a658ea6068\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.811415 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.811341 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:16.966854 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.966831 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x"] Jul 18 01:59:16.968701 ip-10-0-137-108 kubenswrapper[2548]: W0718 01:59:16.968675 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podacfe2150_68c0_4f24_bb4d_73a658ea6068.slice/crio-a6b41eb965564de4d72d73b9f574f0c4016b3e0feb0aca3c6ab0f368dd59ee72 WatchSource:0}: Error finding container a6b41eb965564de4d72d73b9f574f0c4016b3e0feb0aca3c6ab0f368dd59ee72: Status 404 returned error can't find the container with id a6b41eb965564de4d72d73b9f574f0c4016b3e0feb0aca3c6ab0f368dd59ee72 Jul 18 01:59:16.969985 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:16.969969 2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 01:59:17.820197 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:17.820158 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" event={"ID":"acfe2150-68c0-4f24-bb4d-73a658ea6068","Type":"ContainerStarted","Data":"9df2d53613bab0397a4ee9a44801b3fb05890c9d2e83502a6a4dd83009daed5c"} Jul 18 01:59:17.820197 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:17.820202 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" event={"ID":"acfe2150-68c0-4f24-bb4d-73a658ea6068","Type":"ContainerStarted","Data":"a6b41eb965564de4d72d73b9f574f0c4016b3e0feb0aca3c6ab0f368dd59ee72"} Jul 18 01:59:17.837003 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:17.836960 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" podStartSLOduration=1.6757757340000001 podStartE2EDuration="1.836945038s" totalImagesPullingTime="161.169304ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:59:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:59:17.835911618 +0000 UTC m=+666.688654959" watchObservedRunningTime="2026-07-18 01:59:17.836945038 +0000 UTC m=+666.689688383" Jul 18 01:59:26.812241 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:26.812204 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:26.812734 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:26.812301 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:26.828764 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:26.828740 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 01:59:26.877792 ip-10-0-137-108 kubenswrapper[2548]: I0718 01:59:26.877750 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x" Jul 18 02:00:00.150389 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:00.150358 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hk26c"] Jul 18 02:00:00.154128 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:00.154110 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hk26c" Jul 18 02:00:00.156914 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:00.156894 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dkjtj\"" Jul 18 02:00:00.163689 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:00.163666 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hk26c"] Jul 18 02:00:00.165629 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:00.165610 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hk26c" Jul 18 02:00:00.299147 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:00.299123 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hk26c"] Jul 18 02:00:00.301083 ip-10-0-137-108 kubenswrapper[2548]: W0718 02:00:00.301045 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61da4321_a86f_4391_9632_d566b514fad2.slice/crio-2c28f758d74869a26c8c14f6f397005f8d05b6a91660c108389fa14eb1fa9256 WatchSource:0}: Error finding container 2c28f758d74869a26c8c14f6f397005f8d05b6a91660c108389fa14eb1fa9256: Status 404 returned error can't find the container with id 2c28f758d74869a26c8c14f6f397005f8d05b6a91660c108389fa14eb1fa9256 Jul 18 02:00:01.031847 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:01.031815 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hk26c" event={"ID":"61da4321-a86f-4391-9632-d566b514fad2","Type":"ContainerStarted","Data":"2c28f758d74869a26c8c14f6f397005f8d05b6a91660c108389fa14eb1fa9256"} Jul 18 02:00:04.049632 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:04.049535 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hk26c" event={"ID":"61da4321-a86f-4391-9632-d566b514fad2","Type":"ContainerStarted","Data":"50e660887e9fe9752f1de275802ab057da3ab902dd38352843293ef61133f6e7"} Jul 18 02:00:04.065594 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:04.065541 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hk26c" podStartSLOduration=0.899785079 podStartE2EDuration="4.065523983s" totalImagesPullingTime="3.165738904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:00:04.063840915 +0000 UTC m=+712.916584259" watchObservedRunningTime="2026-07-18 02:00:04.065523983 +0000 UTC m=+712.918267337" Jul 18 02:00:05.056104 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:05.056063 2548 generic.go:356] "Generic (PLEG): container finished" podID="61da4321-a86f-4391-9632-d566b514fad2" containerID="50e660887e9fe9752f1de275802ab057da3ab902dd38352843293ef61133f6e7" exitCode=0 Jul 18 02:00:05.056509 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:05.056152 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hk26c" event={"ID":"61da4321-a86f-4391-9632-d566b514fad2","Type":"ContainerDied","Data":"50e660887e9fe9752f1de275802ab057da3ab902dd38352843293ef61133f6e7"} Jul 18 02:00:06.200869 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:06.200846 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hk26c" Jul 18 02:00:07.067386 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:07.067351 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hk26c" event={"ID":"61da4321-a86f-4391-9632-d566b514fad2","Type":"ContainerDied","Data":"2c28f758d74869a26c8c14f6f397005f8d05b6a91660c108389fa14eb1fa9256"} Jul 18 02:00:07.067386 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:07.067379 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739000-hk26c" Jul 18 02:00:07.067623 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:07.067410 2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c28f758d74869a26c8c14f6f397005f8d05b6a91660c108389fa14eb1fa9256" Jul 18 02:00:24.056330 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:24.056295 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5f4847d7ff-k8pfq"] Jul 18 02:00:24.056740 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:24.056711 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61da4321-a86f-4391-9632-d566b514fad2" containerName="cleanup" Jul 18 02:00:24.056740 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:24.056725 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="61da4321-a86f-4391-9632-d566b514fad2" containerName="cleanup" Jul 18 02:00:24.057153 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:24.056819 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="61da4321-a86f-4391-9632-d566b514fad2" containerName="cleanup" Jul 18 02:00:24.060080 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:24.060064 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f4847d7ff-k8pfq" Jul 18 02:00:24.065821 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:24.065800 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5f4847d7ff-k8pfq"] Jul 18 02:00:24.122973 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:24.122948 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1871e809-38a6-44e0-8e6c-a2317d250f6f-tls-cert\") pod \"authorino-5f4847d7ff-k8pfq\" (UID: \"1871e809-38a6-44e0-8e6c-a2317d250f6f\") " pod="kuadrant-system/authorino-5f4847d7ff-k8pfq" Jul 18 02:00:24.123113 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:24.122995 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdpzm\" (UniqueName: \"kubernetes.io/projected/1871e809-38a6-44e0-8e6c-a2317d250f6f-kube-api-access-vdpzm\") pod \"authorino-5f4847d7ff-k8pfq\" (UID: \"1871e809-38a6-44e0-8e6c-a2317d250f6f\") " pod="kuadrant-system/authorino-5f4847d7ff-k8pfq" Jul 18 02:00:24.224445 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:24.224411 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1871e809-38a6-44e0-8e6c-a2317d250f6f-tls-cert\") pod \"authorino-5f4847d7ff-k8pfq\" (UID: \"1871e809-38a6-44e0-8e6c-a2317d250f6f\") " pod="kuadrant-system/authorino-5f4847d7ff-k8pfq" Jul 18 02:00:24.224601 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:24.224463 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdpzm\" (UniqueName: \"kubernetes.io/projected/1871e809-38a6-44e0-8e6c-a2317d250f6f-kube-api-access-vdpzm\") pod \"authorino-5f4847d7ff-k8pfq\" (UID: \"1871e809-38a6-44e0-8e6c-a2317d250f6f\") " pod="kuadrant-system/authorino-5f4847d7ff-k8pfq" Jul 18 02:00:24.226908 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:24.226873 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1871e809-38a6-44e0-8e6c-a2317d250f6f-tls-cert\") pod \"authorino-5f4847d7ff-k8pfq\" (UID: \"1871e809-38a6-44e0-8e6c-a2317d250f6f\") " pod="kuadrant-system/authorino-5f4847d7ff-k8pfq" Jul 18 02:00:24.232181 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:24.232159 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdpzm\" (UniqueName: \"kubernetes.io/projected/1871e809-38a6-44e0-8e6c-a2317d250f6f-kube-api-access-vdpzm\") pod \"authorino-5f4847d7ff-k8pfq\" (UID: \"1871e809-38a6-44e0-8e6c-a2317d250f6f\") " pod="kuadrant-system/authorino-5f4847d7ff-k8pfq" Jul 18 02:00:24.371047 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:24.370959 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5f4847d7ff-k8pfq" Jul 18 02:00:24.513912 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:24.513719 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5f4847d7ff-k8pfq"] Jul 18 02:00:24.516512 ip-10-0-137-108 kubenswrapper[2548]: W0718 02:00:24.516486 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1871e809_38a6_44e0_8e6c_a2317d250f6f.slice/crio-cbb9b438b41ca2184f31ae80325f7f2425d20de817fa6f88a86c97d2ac5e3b1d WatchSource:0}: Error finding container cbb9b438b41ca2184f31ae80325f7f2425d20de817fa6f88a86c97d2ac5e3b1d: Status 404 returned error can't find the container with id cbb9b438b41ca2184f31ae80325f7f2425d20de817fa6f88a86c97d2ac5e3b1d Jul 18 02:00:25.163373 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:25.163336 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f4847d7ff-k8pfq" event={"ID":"1871e809-38a6-44e0-8e6c-a2317d250f6f","Type":"ContainerStarted","Data":"b54a65c2cb0e4479c3e0df0df2800ec92a423b67228b4e5d238601cd6692f204"} Jul 18 02:00:25.163800 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:25.163384 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5f4847d7ff-k8pfq" event={"ID":"1871e809-38a6-44e0-8e6c-a2317d250f6f","Type":"ContainerStarted","Data":"cbb9b438b41ca2184f31ae80325f7f2425d20de817fa6f88a86c97d2ac5e3b1d"} Jul 18 02:00:25.178124 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:25.178071 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5f4847d7ff-k8pfq" podStartSLOduration=0.719749991 podStartE2EDuration="1.17805287s" totalImagesPullingTime="458.302879ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:00:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:00:25.177388789 +0000 UTC m=+734.030132145" watchObservedRunningTime="2026-07-18 02:00:25.17805287 +0000 UTC m=+734.030796214" Jul 18 02:00:25.207058 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:25.206430 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bbbd47994-ts56j"] Jul 18 02:00:25.207058 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:25.206649 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bbbd47994-ts56j" podUID="53925eac-d4eb-49f4-9044-af939affe529" containerName="authorino" containerID="cri-o://0e217f043a7fa3ab6e51de746e3d7884344fa45afb2a6a40198605b6306ed4f8" gracePeriod=30 Jul 18 02:00:25.475704 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:25.475676 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bbbd47994-ts56j" Jul 18 02:00:25.540593 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:25.540505 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jb4t4\" (UniqueName: \"kubernetes.io/projected/53925eac-d4eb-49f4-9044-af939affe529-kube-api-access-jb4t4\") pod \"53925eac-d4eb-49f4-9044-af939affe529\" (UID: \"53925eac-d4eb-49f4-9044-af939affe529\") " Jul 18 02:00:25.540823 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:25.540654 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/53925eac-d4eb-49f4-9044-af939affe529-tls-cert\") pod \"53925eac-d4eb-49f4-9044-af939affe529\" (UID: \"53925eac-d4eb-49f4-9044-af939affe529\") " Jul 18 02:00:25.543238 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:25.543205 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53925eac-d4eb-49f4-9044-af939affe529-kube-api-access-jb4t4" (OuterVolumeSpecName: "kube-api-access-jb4t4") pod "53925eac-d4eb-49f4-9044-af939affe529" (UID: "53925eac-d4eb-49f4-9044-af939affe529"). InnerVolumeSpecName "kube-api-access-jb4t4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:00:25.562893 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:25.562858 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53925eac-d4eb-49f4-9044-af939affe529-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "53925eac-d4eb-49f4-9044-af939affe529" (UID: "53925eac-d4eb-49f4-9044-af939affe529"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:00:25.641922 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:25.641879 2548 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/53925eac-d4eb-49f4-9044-af939affe529-tls-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:00:25.642193 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:25.642162 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jb4t4\" (UniqueName: \"kubernetes.io/projected/53925eac-d4eb-49f4-9044-af939affe529-kube-api-access-jb4t4\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:00:26.172064 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:26.172019 2548 generic.go:356] "Generic (PLEG): container finished" podID="53925eac-d4eb-49f4-9044-af939affe529" containerID="0e217f043a7fa3ab6e51de746e3d7884344fa45afb2a6a40198605b6306ed4f8" exitCode=0 Jul 18 02:00:26.172831 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:26.172807 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bbbd47994-ts56j" Jul 18 02:00:26.176869 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:26.176834 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bbbd47994-ts56j" event={"ID":"53925eac-d4eb-49f4-9044-af939affe529","Type":"ContainerDied","Data":"0e217f043a7fa3ab6e51de746e3d7884344fa45afb2a6a40198605b6306ed4f8"} Jul 18 02:00:26.176971 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:26.176887 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bbbd47994-ts56j" event={"ID":"53925eac-d4eb-49f4-9044-af939affe529","Type":"ContainerDied","Data":"0d0fbcb30af636572f476efdb0758af6a236d5095e1cc3bf703e7762b36b4013"} Jul 18 02:00:26.176971 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:26.176911 2548 scope.go:117] "RemoveContainer" containerID="0e217f043a7fa3ab6e51de746e3d7884344fa45afb2a6a40198605b6306ed4f8" Jul 18 02:00:26.195575 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:26.195556 2548 scope.go:117] "RemoveContainer" containerID="0e217f043a7fa3ab6e51de746e3d7884344fa45afb2a6a40198605b6306ed4f8" Jul 18 02:00:26.195904 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:00:26.195869 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e217f043a7fa3ab6e51de746e3d7884344fa45afb2a6a40198605b6306ed4f8\": container with ID starting with 0e217f043a7fa3ab6e51de746e3d7884344fa45afb2a6a40198605b6306ed4f8 not found: ID does not exist" containerID="0e217f043a7fa3ab6e51de746e3d7884344fa45afb2a6a40198605b6306ed4f8" Jul 18 02:00:26.196025 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:26.195915 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e217f043a7fa3ab6e51de746e3d7884344fa45afb2a6a40198605b6306ed4f8"} err="failed to get container status \"0e217f043a7fa3ab6e51de746e3d7884344fa45afb2a6a40198605b6306ed4f8\": rpc error: code = NotFound desc = could not find container \"0e217f043a7fa3ab6e51de746e3d7884344fa45afb2a6a40198605b6306ed4f8\": container with ID starting with 0e217f043a7fa3ab6e51de746e3d7884344fa45afb2a6a40198605b6306ed4f8 not found: ID does not exist" Jul 18 02:00:26.210162 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:26.210137 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bbbd47994-ts56j"] Jul 18 02:00:26.212087 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:26.212066 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bbbd47994-ts56j"] Jul 18 02:00:27.551736 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:00:27.551696 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53925eac-d4eb-49f4-9044-af939affe529" path="/var/lib/kubelet/pods/53925eac-d4eb-49f4-9044-af939affe529/volumes" Jul 18 02:01:11.069556 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.069513 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75667cc8f5-tswqq"] Jul 18 02:01:11.070173 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.069760 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75667cc8f5-tswqq" podUID="42eef91e-9264-460e-b983-f5c61e6d5f27" containerName="manager" containerID="cri-o://363eb8d5a782937fc0d54defe1c48b46e053f57be1f1119dbe9d4c35ae7d49da" gracePeriod=30 Jul 18 02:01:11.329206 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.329150 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75667cc8f5-tswqq" Jul 18 02:01:11.408365 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.408335 2548 generic.go:356] "Generic (PLEG): container finished" podID="42eef91e-9264-460e-b983-f5c61e6d5f27" containerID="363eb8d5a782937fc0d54defe1c48b46e053f57be1f1119dbe9d4c35ae7d49da" exitCode=0 Jul 18 02:01:11.408539 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.408378 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75667cc8f5-tswqq" Jul 18 02:01:11.408539 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.408419 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75667cc8f5-tswqq" event={"ID":"42eef91e-9264-460e-b983-f5c61e6d5f27","Type":"ContainerDied","Data":"363eb8d5a782937fc0d54defe1c48b46e053f57be1f1119dbe9d4c35ae7d49da"} Jul 18 02:01:11.408539 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.408476 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75667cc8f5-tswqq" event={"ID":"42eef91e-9264-460e-b983-f5c61e6d5f27","Type":"ContainerDied","Data":"f46beb7527c94c3fefe22dea8fb0d5bc1174eb5a0f1d944e18cb52a9377031fb"} Jul 18 02:01:11.408539 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.408500 2548 scope.go:117] "RemoveContainer" containerID="363eb8d5a782937fc0d54defe1c48b46e053f57be1f1119dbe9d4c35ae7d49da" Jul 18 02:01:11.417041 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.417022 2548 scope.go:117] "RemoveContainer" containerID="363eb8d5a782937fc0d54defe1c48b46e053f57be1f1119dbe9d4c35ae7d49da" Jul 18 02:01:11.417315 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:01:11.417297 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"363eb8d5a782937fc0d54defe1c48b46e053f57be1f1119dbe9d4c35ae7d49da\": container with ID starting with 363eb8d5a782937fc0d54defe1c48b46e053f57be1f1119dbe9d4c35ae7d49da not found: ID does not exist" containerID="363eb8d5a782937fc0d54defe1c48b46e053f57be1f1119dbe9d4c35ae7d49da" Jul 18 02:01:11.417366 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.417323 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"363eb8d5a782937fc0d54defe1c48b46e053f57be1f1119dbe9d4c35ae7d49da"} err="failed to get container status \"363eb8d5a782937fc0d54defe1c48b46e053f57be1f1119dbe9d4c35ae7d49da\": rpc error: code = NotFound desc = could not find container \"363eb8d5a782937fc0d54defe1c48b46e053f57be1f1119dbe9d4c35ae7d49da\": container with ID starting with 363eb8d5a782937fc0d54defe1c48b46e053f57be1f1119dbe9d4c35ae7d49da not found: ID does not exist" Jul 18 02:01:11.429595 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.429576 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkfdw\" (UniqueName: \"kubernetes.io/projected/42eef91e-9264-460e-b983-f5c61e6d5f27-kube-api-access-rkfdw\") pod \"42eef91e-9264-460e-b983-f5c61e6d5f27\" (UID: \"42eef91e-9264-460e-b983-f5c61e6d5f27\") " Jul 18 02:01:11.429693 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.429632 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42eef91e-9264-460e-b983-f5c61e6d5f27-webhook-cert\") pod \"42eef91e-9264-460e-b983-f5c61e6d5f27\" (UID: \"42eef91e-9264-460e-b983-f5c61e6d5f27\") " Jul 18 02:01:11.431559 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.431536 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42eef91e-9264-460e-b983-f5c61e6d5f27-kube-api-access-rkfdw" (OuterVolumeSpecName: "kube-api-access-rkfdw") pod "42eef91e-9264-460e-b983-f5c61e6d5f27" (UID: "42eef91e-9264-460e-b983-f5c61e6d5f27"). InnerVolumeSpecName "kube-api-access-rkfdw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:01:11.431709 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.431692 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42eef91e-9264-460e-b983-f5c61e6d5f27-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "42eef91e-9264-460e-b983-f5c61e6d5f27" (UID: "42eef91e-9264-460e-b983-f5c61e6d5f27"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:01:11.531155 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.531130 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkfdw\" (UniqueName: \"kubernetes.io/projected/42eef91e-9264-460e-b983-f5c61e6d5f27-kube-api-access-rkfdw\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:01:11.531155 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.531152 2548 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/42eef91e-9264-460e-b983-f5c61e6d5f27-webhook-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:01:11.724485 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.724456 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75667cc8f5-tswqq"] Jul 18 02:01:11.728276 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:11.728255 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75667cc8f5-tswqq"] Jul 18 02:01:13.549709 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:01:13.549674 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42eef91e-9264-460e-b983-f5c61e6d5f27" path="/var/lib/kubelet/pods/42eef91e-9264-460e-b983-f5c61e6d5f27/volumes" Jul 18 02:02:12.523670 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.523634 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75667cc8f5-s55qn"] Jul 18 02:02:12.524133 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.524036 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42eef91e-9264-460e-b983-f5c61e6d5f27" containerName="manager" Jul 18 02:02:12.524133 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.524047 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="42eef91e-9264-460e-b983-f5c61e6d5f27" containerName="manager" Jul 18 02:02:12.524133 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.524072 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53925eac-d4eb-49f4-9044-af939affe529" containerName="authorino" Jul 18 02:02:12.524133 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.524079 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="53925eac-d4eb-49f4-9044-af939affe529" containerName="authorino" Jul 18 02:02:12.524264 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.524136 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="53925eac-d4eb-49f4-9044-af939affe529" containerName="authorino" Jul 18 02:02:12.524264 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.524146 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="42eef91e-9264-460e-b983-f5c61e6d5f27" containerName="manager" Jul 18 02:02:12.527125 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.527110 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75667cc8f5-s55qn" Jul 18 02:02:12.530503 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.530483 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 18 02:02:12.530641 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.530524 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zpvtf\"" Jul 18 02:02:12.530641 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.530524 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 18 02:02:12.536614 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.536591 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75667cc8f5-s55qn"] Jul 18 02:02:12.565597 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.565565 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b770f0f0-6c75-4004-a41c-4e7f95f91b62-webhook-cert\") pod \"maas-controller-75667cc8f5-s55qn\" (UID: \"b770f0f0-6c75-4004-a41c-4e7f95f91b62\") " pod="opendatahub/maas-controller-75667cc8f5-s55qn" Jul 18 02:02:12.565752 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.565729 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fsgc\" (UniqueName: \"kubernetes.io/projected/b770f0f0-6c75-4004-a41c-4e7f95f91b62-kube-api-access-8fsgc\") pod \"maas-controller-75667cc8f5-s55qn\" (UID: \"b770f0f0-6c75-4004-a41c-4e7f95f91b62\") " pod="opendatahub/maas-controller-75667cc8f5-s55qn" Jul 18 02:02:12.667007 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.666979 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fsgc\" (UniqueName: \"kubernetes.io/projected/b770f0f0-6c75-4004-a41c-4e7f95f91b62-kube-api-access-8fsgc\") pod \"maas-controller-75667cc8f5-s55qn\" (UID: \"b770f0f0-6c75-4004-a41c-4e7f95f91b62\") " pod="opendatahub/maas-controller-75667cc8f5-s55qn" Jul 18 02:02:12.667151 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.667020 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b770f0f0-6c75-4004-a41c-4e7f95f91b62-webhook-cert\") pod \"maas-controller-75667cc8f5-s55qn\" (UID: \"b770f0f0-6c75-4004-a41c-4e7f95f91b62\") " pod="opendatahub/maas-controller-75667cc8f5-s55qn" Jul 18 02:02:12.669341 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.669311 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b770f0f0-6c75-4004-a41c-4e7f95f91b62-webhook-cert\") pod \"maas-controller-75667cc8f5-s55qn\" (UID: \"b770f0f0-6c75-4004-a41c-4e7f95f91b62\") " pod="opendatahub/maas-controller-75667cc8f5-s55qn" Jul 18 02:02:12.676489 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.676466 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fsgc\" (UniqueName: \"kubernetes.io/projected/b770f0f0-6c75-4004-a41c-4e7f95f91b62-kube-api-access-8fsgc\") pod \"maas-controller-75667cc8f5-s55qn\" (UID: \"b770f0f0-6c75-4004-a41c-4e7f95f91b62\") " pod="opendatahub/maas-controller-75667cc8f5-s55qn" Jul 18 02:02:12.839060 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.839004 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75667cc8f5-s55qn" Jul 18 02:02:12.976593 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:12.976569 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75667cc8f5-s55qn"] Jul 18 02:02:12.978294 ip-10-0-137-108 kubenswrapper[2548]: W0718 02:02:12.978258 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb770f0f0_6c75_4004_a41c_4e7f95f91b62.slice/crio-1d1881c0ba9277daaf5821285da69bc2668f416fdb7e741c8fcdf7bb42f08018 WatchSource:0}: Error finding container 1d1881c0ba9277daaf5821285da69bc2668f416fdb7e741c8fcdf7bb42f08018: Status 404 returned error can't find the container with id 1d1881c0ba9277daaf5821285da69bc2668f416fdb7e741c8fcdf7bb42f08018 Jul 18 02:02:13.719584 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:13.719549 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75667cc8f5-s55qn" event={"ID":"b770f0f0-6c75-4004-a41c-4e7f95f91b62","Type":"ContainerStarted","Data":"1d1881c0ba9277daaf5821285da69bc2668f416fdb7e741c8fcdf7bb42f08018"} Jul 18 02:02:14.726156 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:14.726114 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75667cc8f5-s55qn" event={"ID":"b770f0f0-6c75-4004-a41c-4e7f95f91b62","Type":"ContainerStarted","Data":"e94d487321dec7f28a537a2e24edae6b8da7de9257224b5e5439a76057394613"} Jul 18 02:02:14.726560 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:14.726261 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75667cc8f5-s55qn" Jul 18 02:02:14.744281 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:14.744236 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75667cc8f5-s55qn" podStartSLOduration=1.61942129 podStartE2EDuration="2.744221923s" totalImagesPullingTime="1.124800633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:02:14.74172388 +0000 UTC m=+843.594467226" watchObservedRunningTime="2026-07-18 02:02:14.744221923 +0000 UTC m=+843.596965316" Jul 18 02:02:25.733999 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:25.733965 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75667cc8f5-s55qn" Jul 18 02:02:30.415813 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:30.415755 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75667cc8f5-s55qn"] Jul 18 02:02:30.416295 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:30.415988 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75667cc8f5-s55qn" podUID="b770f0f0-6c75-4004-a41c-4e7f95f91b62" containerName="manager" containerID="cri-o://e94d487321dec7f28a537a2e24edae6b8da7de9257224b5e5439a76057394613" gracePeriod=30 Jul 18 02:02:30.670003 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:30.669947 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75667cc8f5-s55qn" Jul 18 02:02:30.805113 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:30.805074 2548 generic.go:356] "Generic (PLEG): container finished" podID="b770f0f0-6c75-4004-a41c-4e7f95f91b62" containerID="e94d487321dec7f28a537a2e24edae6b8da7de9257224b5e5439a76057394613" exitCode=0 Jul 18 02:02:30.805303 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:30.805117 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75667cc8f5-s55qn" Jul 18 02:02:30.805303 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:30.805160 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75667cc8f5-s55qn" event={"ID":"b770f0f0-6c75-4004-a41c-4e7f95f91b62","Type":"ContainerDied","Data":"e94d487321dec7f28a537a2e24edae6b8da7de9257224b5e5439a76057394613"} Jul 18 02:02:30.805303 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:30.805214 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75667cc8f5-s55qn" event={"ID":"b770f0f0-6c75-4004-a41c-4e7f95f91b62","Type":"ContainerDied","Data":"1d1881c0ba9277daaf5821285da69bc2668f416fdb7e741c8fcdf7bb42f08018"} Jul 18 02:02:30.805303 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:30.805243 2548 scope.go:117] "RemoveContainer" containerID="e94d487321dec7f28a537a2e24edae6b8da7de9257224b5e5439a76057394613" Jul 18 02:02:30.813606 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:30.813589 2548 scope.go:117] "RemoveContainer" containerID="e94d487321dec7f28a537a2e24edae6b8da7de9257224b5e5439a76057394613" Jul 18 02:02:30.813760 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:30.813741 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b770f0f0-6c75-4004-a41c-4e7f95f91b62-webhook-cert\") pod \"b770f0f0-6c75-4004-a41c-4e7f95f91b62\" (UID: \"b770f0f0-6c75-4004-a41c-4e7f95f91b62\") " Jul 18 02:02:30.813887 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:02:30.813862 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e94d487321dec7f28a537a2e24edae6b8da7de9257224b5e5439a76057394613\": container with ID starting with e94d487321dec7f28a537a2e24edae6b8da7de9257224b5e5439a76057394613 not found: ID does not exist" containerID="e94d487321dec7f28a537a2e24edae6b8da7de9257224b5e5439a76057394613" Jul 18 02:02:30.813887 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:30.813872 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fsgc\" (UniqueName: \"kubernetes.io/projected/b770f0f0-6c75-4004-a41c-4e7f95f91b62-kube-api-access-8fsgc\") pod \"b770f0f0-6c75-4004-a41c-4e7f95f91b62\" (UID: \"b770f0f0-6c75-4004-a41c-4e7f95f91b62\") " Jul 18 02:02:30.813972 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:30.813887 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e94d487321dec7f28a537a2e24edae6b8da7de9257224b5e5439a76057394613"} err="failed to get container status \"e94d487321dec7f28a537a2e24edae6b8da7de9257224b5e5439a76057394613\": rpc error: code = NotFound desc = could not find container \"e94d487321dec7f28a537a2e24edae6b8da7de9257224b5e5439a76057394613\": container with ID starting with e94d487321dec7f28a537a2e24edae6b8da7de9257224b5e5439a76057394613 not found: ID does not exist" Jul 18 02:02:30.815703 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:30.815677 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b770f0f0-6c75-4004-a41c-4e7f95f91b62-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b770f0f0-6c75-4004-a41c-4e7f95f91b62" (UID: "b770f0f0-6c75-4004-a41c-4e7f95f91b62"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 18 02:02:30.816064 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:30.816043 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b770f0f0-6c75-4004-a41c-4e7f95f91b62-kube-api-access-8fsgc" (OuterVolumeSpecName: "kube-api-access-8fsgc") pod "b770f0f0-6c75-4004-a41c-4e7f95f91b62" (UID: "b770f0f0-6c75-4004-a41c-4e7f95f91b62"). InnerVolumeSpecName "kube-api-access-8fsgc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:02:30.914696 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:30.914659 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fsgc\" (UniqueName: \"kubernetes.io/projected/b770f0f0-6c75-4004-a41c-4e7f95f91b62-kube-api-access-8fsgc\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:02:30.914696 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:30.914692 2548 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b770f0f0-6c75-4004-a41c-4e7f95f91b62-webhook-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:02:31.127765 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:31.127737 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75667cc8f5-s55qn"] Jul 18 02:02:31.131626 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:31.131601 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75667cc8f5-s55qn"] Jul 18 02:02:31.548928 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:31.548900 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b770f0f0-6c75-4004-a41c-4e7f95f91b62" path="/var/lib/kubelet/pods/b770f0f0-6c75-4004-a41c-4e7f95f91b62/volumes" Jul 18 02:02:31.886566 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:31.886467 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75667cc8f5-q7hjf"] Jul 18 02:02:31.886919 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:31.886906 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b770f0f0-6c75-4004-a41c-4e7f95f91b62" containerName="manager" Jul 18 02:02:31.886974 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:31.886920 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="b770f0f0-6c75-4004-a41c-4e7f95f91b62" containerName="manager" Jul 18 02:02:31.887008 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:31.886997 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="b770f0f0-6c75-4004-a41c-4e7f95f91b62" containerName="manager" Jul 18 02:02:31.891725 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:31.891700 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75667cc8f5-q7hjf" Jul 18 02:02:31.894076 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:31.894058 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 18 02:02:31.895037 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:31.895013 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 18 02:02:31.895496 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:31.895061 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zpvtf\"" Jul 18 02:02:31.897868 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:31.897848 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75667cc8f5-q7hjf"] Jul 18 02:02:32.024229 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:32.024192 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1fa4f31e-6ff9-4085-b65d-f8c04aa7d17a-webhook-cert\") pod \"maas-controller-75667cc8f5-q7hjf\" (UID: \"1fa4f31e-6ff9-4085-b65d-f8c04aa7d17a\") " pod="opendatahub/maas-controller-75667cc8f5-q7hjf" Jul 18 02:02:32.024403 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:32.024236 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w89f9\" (UniqueName: \"kubernetes.io/projected/1fa4f31e-6ff9-4085-b65d-f8c04aa7d17a-kube-api-access-w89f9\") pod \"maas-controller-75667cc8f5-q7hjf\" (UID: \"1fa4f31e-6ff9-4085-b65d-f8c04aa7d17a\") " pod="opendatahub/maas-controller-75667cc8f5-q7hjf" Jul 18 02:02:32.125617 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:32.125582 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1fa4f31e-6ff9-4085-b65d-f8c04aa7d17a-webhook-cert\") pod \"maas-controller-75667cc8f5-q7hjf\" (UID: \"1fa4f31e-6ff9-4085-b65d-f8c04aa7d17a\") " pod="opendatahub/maas-controller-75667cc8f5-q7hjf" Jul 18 02:02:32.125617 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:32.125622 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w89f9\" (UniqueName: \"kubernetes.io/projected/1fa4f31e-6ff9-4085-b65d-f8c04aa7d17a-kube-api-access-w89f9\") pod \"maas-controller-75667cc8f5-q7hjf\" (UID: \"1fa4f31e-6ff9-4085-b65d-f8c04aa7d17a\") " pod="opendatahub/maas-controller-75667cc8f5-q7hjf" Jul 18 02:02:32.128089 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:32.128065 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1fa4f31e-6ff9-4085-b65d-f8c04aa7d17a-webhook-cert\") pod \"maas-controller-75667cc8f5-q7hjf\" (UID: \"1fa4f31e-6ff9-4085-b65d-f8c04aa7d17a\") " pod="opendatahub/maas-controller-75667cc8f5-q7hjf" Jul 18 02:02:32.133435 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:32.133411 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w89f9\" (UniqueName: \"kubernetes.io/projected/1fa4f31e-6ff9-4085-b65d-f8c04aa7d17a-kube-api-access-w89f9\") pod \"maas-controller-75667cc8f5-q7hjf\" (UID: \"1fa4f31e-6ff9-4085-b65d-f8c04aa7d17a\") " pod="opendatahub/maas-controller-75667cc8f5-q7hjf" Jul 18 02:02:32.204215 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:32.204191 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75667cc8f5-q7hjf" Jul 18 02:02:32.342590 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:32.342563 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75667cc8f5-q7hjf"] Jul 18 02:02:32.345206 ip-10-0-137-108 kubenswrapper[2548]: W0718 02:02:32.345176 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fa4f31e_6ff9_4085_b65d_f8c04aa7d17a.slice/crio-dd7cb86ad9fb358a6d1d96835a5eb0a290d9f75178cdbfa12aafa90a566e701c WatchSource:0}: Error finding container dd7cb86ad9fb358a6d1d96835a5eb0a290d9f75178cdbfa12aafa90a566e701c: Status 404 returned error can't find the container with id dd7cb86ad9fb358a6d1d96835a5eb0a290d9f75178cdbfa12aafa90a566e701c Jul 18 02:02:32.818266 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:32.818232 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75667cc8f5-q7hjf" event={"ID":"1fa4f31e-6ff9-4085-b65d-f8c04aa7d17a","Type":"ContainerStarted","Data":"4cbef1376100996759f6ede4c75b02cf3aae6e7ec2ac6e993e3734b8b7834320"} Jul 18 02:02:32.818637 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:32.818292 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75667cc8f5-q7hjf" Jul 18 02:02:32.818637 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:32.818314 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75667cc8f5-q7hjf" event={"ID":"1fa4f31e-6ff9-4085-b65d-f8c04aa7d17a","Type":"ContainerStarted","Data":"dd7cb86ad9fb358a6d1d96835a5eb0a290d9f75178cdbfa12aafa90a566e701c"} Jul 18 02:02:32.834765 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:32.834719 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75667cc8f5-q7hjf" podStartSLOduration=1.542096057 podStartE2EDuration="1.834705636s" totalImagesPullingTime="292.609579ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:02:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:02:32.833185416 +0000 UTC m=+861.685928761" watchObservedRunningTime="2026-07-18 02:02:32.834705636 +0000 UTC m=+861.687448980" Jul 18 02:02:43.825092 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:02:43.825063 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75667cc8f5-q7hjf" Jul 18 02:03:11.523816 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:03:11.523721 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 02:03:11.524376 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:03:11.523895 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 02:07:19.732596 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:19.732560 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78"] Jul 18 02:07:19.733177 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:19.732771 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" podUID="d60a599b-65ac-4991-b2e4-a74387659a39" containerName="manager" containerID="cri-o://482effd2549a36a3633ce7f31b5ed579441eb9c2b95a8cea046a79e92e8aae01" gracePeriod=10 Jul 18 02:07:23.191383 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:23.191355 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" Jul 18 02:07:23.302629 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:23.302536 2548 generic.go:356] "Generic (PLEG): container finished" podID="d60a599b-65ac-4991-b2e4-a74387659a39" containerID="482effd2549a36a3633ce7f31b5ed579441eb9c2b95a8cea046a79e92e8aae01" exitCode=0 Jul 18 02:07:23.302629 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:23.302585 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" Jul 18 02:07:23.302845 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:23.302584 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" event={"ID":"d60a599b-65ac-4991-b2e4-a74387659a39","Type":"ContainerDied","Data":"482effd2549a36a3633ce7f31b5ed579441eb9c2b95a8cea046a79e92e8aae01"} Jul 18 02:07:23.302845 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:23.302720 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78" event={"ID":"d60a599b-65ac-4991-b2e4-a74387659a39","Type":"ContainerDied","Data":"44ea22744b31d172825373dcd6742ecf50cf74cb587850a43918464fedd400bb"} Jul 18 02:07:23.302845 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:23.302749 2548 scope.go:117] "RemoveContainer" containerID="482effd2549a36a3633ce7f31b5ed579441eb9c2b95a8cea046a79e92e8aae01" Jul 18 02:07:23.312157 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:23.312142 2548 scope.go:117] "RemoveContainer" containerID="482effd2549a36a3633ce7f31b5ed579441eb9c2b95a8cea046a79e92e8aae01" Jul 18 02:07:23.312434 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:07:23.312408 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"482effd2549a36a3633ce7f31b5ed579441eb9c2b95a8cea046a79e92e8aae01\": container with ID starting with 482effd2549a36a3633ce7f31b5ed579441eb9c2b95a8cea046a79e92e8aae01 not found: ID does not exist" containerID="482effd2549a36a3633ce7f31b5ed579441eb9c2b95a8cea046a79e92e8aae01" Jul 18 02:07:23.312528 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:23.312443 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"482effd2549a36a3633ce7f31b5ed579441eb9c2b95a8cea046a79e92e8aae01"} err="failed to get container status \"482effd2549a36a3633ce7f31b5ed579441eb9c2b95a8cea046a79e92e8aae01\": rpc error: code = NotFound desc = could not find container \"482effd2549a36a3633ce7f31b5ed579441eb9c2b95a8cea046a79e92e8aae01\": container with ID starting with 482effd2549a36a3633ce7f31b5ed579441eb9c2b95a8cea046a79e92e8aae01 not found: ID does not exist" Jul 18 02:07:23.341792 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:23.341761 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5sxq\" (UniqueName: \"kubernetes.io/projected/d60a599b-65ac-4991-b2e4-a74387659a39-kube-api-access-s5sxq\") pod \"d60a599b-65ac-4991-b2e4-a74387659a39\" (UID: \"d60a599b-65ac-4991-b2e4-a74387659a39\") " Jul 18 02:07:23.341904 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:23.341829 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d60a599b-65ac-4991-b2e4-a74387659a39-extensions-socket-volume\") pod \"d60a599b-65ac-4991-b2e4-a74387659a39\" (UID: \"d60a599b-65ac-4991-b2e4-a74387659a39\") " Jul 18 02:07:23.342179 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:23.342155 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d60a599b-65ac-4991-b2e4-a74387659a39-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d60a599b-65ac-4991-b2e4-a74387659a39" (UID: "d60a599b-65ac-4991-b2e4-a74387659a39"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:07:23.343655 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:23.343629 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d60a599b-65ac-4991-b2e4-a74387659a39-kube-api-access-s5sxq" (OuterVolumeSpecName: "kube-api-access-s5sxq") pod "d60a599b-65ac-4991-b2e4-a74387659a39" (UID: "d60a599b-65ac-4991-b2e4-a74387659a39"). InnerVolumeSpecName "kube-api-access-s5sxq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:07:23.443453 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:23.443431 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5sxq\" (UniqueName: \"kubernetes.io/projected/d60a599b-65ac-4991-b2e4-a74387659a39-kube-api-access-s5sxq\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:07:23.443453 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:23.443451 2548 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d60a599b-65ac-4991-b2e4-a74387659a39-extensions-socket-volume\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:07:23.617767 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:23.617737 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78"] Jul 18 02:07:23.621990 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:23.621968 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g6p78"] Jul 18 02:07:25.548509 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:07:25.548476 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d60a599b-65ac-4991-b2e4-a74387659a39" path="/var/lib/kubelet/pods/d60a599b-65ac-4991-b2e4-a74387659a39/volumes" Jul 18 02:08:11.554518 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:11.554487 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 02:08:11.556329 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:11.556307 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 02:08:20.881720 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:20.881688 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hfmw"] Jul 18 02:08:20.882182 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:20.882090 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d60a599b-65ac-4991-b2e4-a74387659a39" containerName="manager" Jul 18 02:08:20.882182 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:20.882102 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="d60a599b-65ac-4991-b2e4-a74387659a39" containerName="manager" Jul 18 02:08:20.882182 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:20.882182 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="d60a599b-65ac-4991-b2e4-a74387659a39" containerName="manager" Jul 18 02:08:20.886375 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:20.886357 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hfmw" Jul 18 02:08:20.888953 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:20.888931 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-qsppv\"" Jul 18 02:08:20.897136 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:20.897112 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hfmw"] Jul 18 02:08:21.032995 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:21.032965 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqz2c\" (UniqueName: \"kubernetes.io/projected/7b2878f6-42fb-4b8b-bc1f-d2102423d6ec-kube-api-access-jqz2c\") pod \"kuadrant-operator-controller-manager-86878c769f-9hfmw\" (UID: \"7b2878f6-42fb-4b8b-bc1f-d2102423d6ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hfmw" Jul 18 02:08:21.033147 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:21.033048 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b2878f6-42fb-4b8b-bc1f-d2102423d6ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9hfmw\" (UID: \"7b2878f6-42fb-4b8b-bc1f-d2102423d6ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hfmw" Jul 18 02:08:21.134242 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:21.134178 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqz2c\" (UniqueName: \"kubernetes.io/projected/7b2878f6-42fb-4b8b-bc1f-d2102423d6ec-kube-api-access-jqz2c\") pod \"kuadrant-operator-controller-manager-86878c769f-9hfmw\" (UID: \"7b2878f6-42fb-4b8b-bc1f-d2102423d6ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hfmw" Jul 18 02:08:21.134368 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:21.134253 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b2878f6-42fb-4b8b-bc1f-d2102423d6ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9hfmw\" (UID: \"7b2878f6-42fb-4b8b-bc1f-d2102423d6ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hfmw" Jul 18 02:08:21.134584 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:21.134569 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b2878f6-42fb-4b8b-bc1f-d2102423d6ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9hfmw\" (UID: \"7b2878f6-42fb-4b8b-bc1f-d2102423d6ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hfmw" Jul 18 02:08:21.143059 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:21.143038 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqz2c\" (UniqueName: \"kubernetes.io/projected/7b2878f6-42fb-4b8b-bc1f-d2102423d6ec-kube-api-access-jqz2c\") pod \"kuadrant-operator-controller-manager-86878c769f-9hfmw\" (UID: \"7b2878f6-42fb-4b8b-bc1f-d2102423d6ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hfmw" Jul 18 02:08:21.196466 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:21.196444 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hfmw" Jul 18 02:08:21.339281 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:21.339259 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hfmw"] Jul 18 02:08:21.342245 ip-10-0-137-108 kubenswrapper[2548]: W0718 02:08:21.342219 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b2878f6_42fb_4b8b_bc1f_d2102423d6ec.slice/crio-f631318424e9b4d6825d4ac0613b21862c9d435851e933a24e0953040e476881 WatchSource:0}: Error finding container f631318424e9b4d6825d4ac0613b21862c9d435851e933a24e0953040e476881: Status 404 returned error can't find the container with id f631318424e9b4d6825d4ac0613b21862c9d435851e933a24e0953040e476881 Jul 18 02:08:21.588308 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:21.588269 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hfmw" event={"ID":"7b2878f6-42fb-4b8b-bc1f-d2102423d6ec","Type":"ContainerStarted","Data":"e1a26e8307467fb3e591bafa4ffa3b4b4db77f5e299e9565d01dc2b3429ea125"} Jul 18 02:08:21.588473 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:21.588334 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hfmw" Jul 18 02:08:21.588473 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:21.588358 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hfmw" event={"ID":"7b2878f6-42fb-4b8b-bc1f-d2102423d6ec","Type":"ContainerStarted","Data":"f631318424e9b4d6825d4ac0613b21862c9d435851e933a24e0953040e476881"} Jul 18 02:08:21.610097 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:21.610052 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hfmw" podStartSLOduration=1.610039411 podStartE2EDuration="1.610039411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:08:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:08:21.609860004 +0000 UTC m=+1210.462603350" watchObservedRunningTime="2026-07-18 02:08:21.610039411 +0000 UTC m=+1210.462782755" Jul 18 02:08:32.596547 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:08:32.596518 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9hfmw" Jul 18 02:13:11.602798 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:13:11.601798 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 02:13:11.626631 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:13:11.626594 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 02:15:00.133064 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:00.133029 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739015-77zdn"] Jul 18 02:15:00.137126 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:00.137111 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-77zdn" Jul 18 02:15:00.140065 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:00.140047 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dkjtj\"" Jul 18 02:15:00.147139 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:00.147111 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739015-77zdn"] Jul 18 02:15:00.149544 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:00.149277 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-77zdn" Jul 18 02:15:00.287644 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:00.287573 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739015-77zdn"] Jul 18 02:15:00.290068 ip-10-0-137-108 kubenswrapper[2548]: W0718 02:15:00.290030 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b63bca_e55c_4e6c_8043_25a8ac2fa33d.slice/crio-21d1b2e4694c2017dbee677725c50405290b956ec5034594762d96e242282a3e WatchSource:0}: Error finding container 21d1b2e4694c2017dbee677725c50405290b956ec5034594762d96e242282a3e: Status 404 returned error can't find the container with id 21d1b2e4694c2017dbee677725c50405290b956ec5034594762d96e242282a3e Jul 18 02:15:00.660842 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:00.660806 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-77zdn" event={"ID":"95b63bca-e55c-4e6c-8043-25a8ac2fa33d","Type":"ContainerStarted","Data":"ea1098a7d4a7afb62fb1fbfd2f85b38332dbe34f1ee0e84e395cb76586d03d63"} Jul 18 02:15:00.661010 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:00.660862 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-77zdn" event={"ID":"95b63bca-e55c-4e6c-8043-25a8ac2fa33d","Type":"ContainerStarted","Data":"21d1b2e4694c2017dbee677725c50405290b956ec5034594762d96e242282a3e"} Jul 18 02:15:00.676144 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:00.676098 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-77zdn" podStartSLOduration=0.676083721 podStartE2EDuration="676.083721ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:15:00.674221837 +0000 UTC m=+1609.526965182" watchObservedRunningTime="2026-07-18 02:15:00.676083721 +0000 UTC m=+1609.528827065" Jul 18 02:15:01.671275 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:01.671242 2548 generic.go:356] "Generic (PLEG): container finished" podID="95b63bca-e55c-4e6c-8043-25a8ac2fa33d" containerID="ea1098a7d4a7afb62fb1fbfd2f85b38332dbe34f1ee0e84e395cb76586d03d63" exitCode=0 Jul 18 02:15:01.671275 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:01.671280 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-77zdn" event={"ID":"95b63bca-e55c-4e6c-8043-25a8ac2fa33d","Type":"ContainerDied","Data":"ea1098a7d4a7afb62fb1fbfd2f85b38332dbe34f1ee0e84e395cb76586d03d63"} Jul 18 02:15:02.820138 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:02.820113 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-77zdn" Jul 18 02:15:03.682413 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:03.682374 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-77zdn" event={"ID":"95b63bca-e55c-4e6c-8043-25a8ac2fa33d","Type":"ContainerDied","Data":"21d1b2e4694c2017dbee677725c50405290b956ec5034594762d96e242282a3e"} Jul 18 02:15:03.682609 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:03.682439 2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21d1b2e4694c2017dbee677725c50405290b956ec5034594762d96e242282a3e" Jul 18 02:15:03.682609 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:03.682397 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739015-77zdn" Jul 18 02:15:24.443406 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:24.443368 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff"] Jul 18 02:15:24.443850 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:24.443815 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95b63bca-e55c-4e6c-8043-25a8ac2fa33d" containerName="cleanup" Jul 18 02:15:24.443850 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:24.443830 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="95b63bca-e55c-4e6c-8043-25a8ac2fa33d" containerName="cleanup" Jul 18 02:15:24.443939 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:24.443913 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="95b63bca-e55c-4e6c-8043-25a8ac2fa33d" containerName="cleanup" Jul 18 02:15:24.447124 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:24.447094 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff" Jul 18 02:15:24.449465 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:24.449443 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dkjtj\"" Jul 18 02:15:24.453972 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:24.453936 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff"] Jul 18 02:15:24.530611 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:24.530578 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/297884c3-3597-4383-9324-f4e541c51afd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff\" (UID: \"297884c3-3597-4383-9324-f4e541c51afd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff" Jul 18 02:15:24.631076 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:24.631047 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/297884c3-3597-4383-9324-f4e541c51afd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff\" (UID: \"297884c3-3597-4383-9324-f4e541c51afd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff" Jul 18 02:15:24.631610 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:24.631592 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/297884c3-3597-4383-9324-f4e541c51afd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff\" (UID: \"297884c3-3597-4383-9324-f4e541c51afd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff" Jul 18 02:15:24.759505 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:24.759410 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff" Jul 18 02:15:25.107449 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:25.107236 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff"] Jul 18 02:15:25.110296 ip-10-0-137-108 kubenswrapper[2548]: W0718 02:15:25.110264 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod297884c3_3597_4383_9324_f4e541c51afd.slice/crio-b59de03320b3d05f9a8c9357fec4ed03604d0372cff448d3ccbf65d2bd5ebfca WatchSource:0}: Error finding container b59de03320b3d05f9a8c9357fec4ed03604d0372cff448d3ccbf65d2bd5ebfca: Status 404 returned error can't find the container with id b59de03320b3d05f9a8c9357fec4ed03604d0372cff448d3ccbf65d2bd5ebfca Jul 18 02:15:25.797087 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:25.797049 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff" event={"ID":"297884c3-3597-4383-9324-f4e541c51afd","Type":"ContainerStarted","Data":"af3bcd6162905ab4f51913e010e555ace7be7ad1389b63694c00b2a91743255f"} Jul 18 02:15:25.797529 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:25.797127 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff" event={"ID":"297884c3-3597-4383-9324-f4e541c51afd","Type":"ContainerStarted","Data":"b59de03320b3d05f9a8c9357fec4ed03604d0372cff448d3ccbf65d2bd5ebfca"} Jul 18 02:15:26.803337 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:26.803295 2548 generic.go:356] "Generic (PLEG): container finished" podID="297884c3-3597-4383-9324-f4e541c51afd" containerID="af3bcd6162905ab4f51913e010e555ace7be7ad1389b63694c00b2a91743255f" exitCode=0 Jul 18 02:15:26.803713 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:26.803374 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff" event={"ID":"297884c3-3597-4383-9324-f4e541c51afd","Type":"ContainerDied","Data":"af3bcd6162905ab4f51913e010e555ace7be7ad1389b63694c00b2a91743255f"} Jul 18 02:15:27.940430 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:27.940407 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff" Jul 18 02:15:28.064082 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:28.064006 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/297884c3-3597-4383-9324-f4e541c51afd-maas-api-service-ca\") pod \"297884c3-3597-4383-9324-f4e541c51afd\" (UID: \"297884c3-3597-4383-9324-f4e541c51afd\") " Jul 18 02:15:28.064345 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:28.064324 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/297884c3-3597-4383-9324-f4e541c51afd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "297884c3-3597-4383-9324-f4e541c51afd" (UID: "297884c3-3597-4383-9324-f4e541c51afd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:15:28.165175 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:28.165151 2548 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/297884c3-3597-4383-9324-f4e541c51afd-maas-api-service-ca\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:15:28.814557 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:28.814520 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff" Jul 18 02:15:28.814716 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:28.814576 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff" event={"ID":"297884c3-3597-4383-9324-f4e541c51afd","Type":"ContainerDied","Data":"b59de03320b3d05f9a8c9357fec4ed03604d0372cff448d3ccbf65d2bd5ebfca"} Jul 18 02:15:28.814716 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:28.814628 2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b59de03320b3d05f9a8c9357fec4ed03604d0372cff448d3ccbf65d2bd5ebfca" Jul 18 02:15:39.506506 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:39.506451 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff"] Jul 18 02:15:39.509005 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:39.508984 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c62b43cb-lc5ff"] Jul 18 02:15:39.549506 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:39.549470 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="297884c3-3597-4383-9324-f4e541c51afd" path="/var/lib/kubelet/pods/297884c3-3597-4383-9324-f4e541c51afd/volumes" Jul 18 02:15:40.633425 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.633394 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf"] Jul 18 02:15:40.633846 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.633832 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="297884c3-3597-4383-9324-f4e541c51afd" containerName="revoke-keys" Jul 18 02:15:40.633846 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.633848 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="297884c3-3597-4383-9324-f4e541c51afd" containerName="revoke-keys" Jul 18 02:15:40.633929 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.633920 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="297884c3-3597-4383-9324-f4e541c51afd" containerName="revoke-keys" Jul 18 02:15:40.637343 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.637329 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.640312 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.640282 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-23b07985-gw-openshift-default-dockercfg-kt9b6\"" Jul 18 02:15:40.659122 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.659095 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf"] Jul 18 02:15:40.678325 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.678299 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-envoy\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.678478 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.678343 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-data\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.678478 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.678421 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-workload-socket\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.678601 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.678474 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-workload-certs\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.678601 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.678504 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grsjk\" (UniqueName: \"kubernetes.io/projected/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-kube-api-access-grsjk\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.678689 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.678622 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-credential-socket\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.678689 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.678666 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-podinfo\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.678849 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.678831 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-token\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.678902 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.678862 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istiod-ca-cert\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.779862 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.779826 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-token\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.779862 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.779863 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istiod-ca-cert\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.780086 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.779895 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-envoy\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.780086 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.779963 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-data\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.780086 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.780014 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-workload-socket\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.780086 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.780050 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-workload-certs\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.780086 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.780075 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grsjk\" (UniqueName: \"kubernetes.io/projected/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-kube-api-access-grsjk\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.780346 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.780110 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-credential-socket\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.780346 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.780142 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-podinfo\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.780741 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.780365 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-data\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.780741 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.780410 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-workload-socket\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.780741 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.780526 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-workload-certs\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.780741 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.780671 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-credential-socket\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.780952 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.780772 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istiod-ca-cert\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.782176 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.782159 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-envoy\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.782623 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.782608 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-podinfo\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.792286 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.792265 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grsjk\" (UniqueName: \"kubernetes.io/projected/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-kube-api-access-grsjk\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.794073 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.794049 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-token\") pod \"e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:40.948085 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:40.948050 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:41.094689 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:41.094666 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf"] Jul 18 02:15:41.096160 ip-10-0-137-108 kubenswrapper[2548]: W0718 02:15:41.096137 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4446c38_3abf_41a4_8e81_c9bcfa02ffc9.slice/crio-7b07ab0b0ea58d2dfbbb0ed57600d99893c99800503b0099f2f01e83a3fa04e8 WatchSource:0}: Error finding container 7b07ab0b0ea58d2dfbbb0ed57600d99893c99800503b0099f2f01e83a3fa04e8: Status 404 returned error can't find the container with id 7b07ab0b0ea58d2dfbbb0ed57600d99893c99800503b0099f2f01e83a3fa04e8 Jul 18 02:15:41.098310 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:41.098281 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:15:41.098371 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:41.098361 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:15:41.098414 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:41.098398 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:15:41.890975 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:41.890941 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" event={"ID":"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9","Type":"ContainerStarted","Data":"0ce59ab2215ca2336588fb28aaeb958f751d22ab2eda3d91bd799fdb1d804639"} Jul 18 02:15:41.891327 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:41.890998 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" event={"ID":"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9","Type":"ContainerStarted","Data":"7b07ab0b0ea58d2dfbbb0ed57600d99893c99800503b0099f2f01e83a3fa04e8"} Jul 18 02:15:41.912129 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:41.912080 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" podStartSLOduration=1.9120646959999998 podStartE2EDuration="1.912064696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:15:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:15:41.909842644 +0000 UTC m=+1650.762585988" watchObservedRunningTime="2026-07-18 02:15:41.912064696 +0000 UTC m=+1650.764808091" Jul 18 02:15:41.948131 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:41.948103 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:42.953928 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:42.953898 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:43.901701 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:43.901666 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:43.902695 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:43.902679 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:57.983519 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:57.983486 2548 generic.go:356] "Generic (PLEG): container finished" podID="f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" containerID="0ce59ab2215ca2336588fb28aaeb958f751d22ab2eda3d91bd799fdb1d804639" exitCode=0 Jul 18 02:15:57.983899 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:57.983561 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" event={"ID":"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9","Type":"ContainerDied","Data":"0ce59ab2215ca2336588fb28aaeb958f751d22ab2eda3d91bd799fdb1d804639"} Jul 18 02:15:57.983958 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:57.983944 2548 scope.go:117] "RemoveContainer" containerID="0ce59ab2215ca2336588fb28aaeb958f751d22ab2eda3d91bd799fdb1d804639" Jul 18 02:15:57.984930 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:57.984903 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:15:57.985013 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:57.984970 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:15:57.985013 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:57.985003 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:15:58.902559 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:58.902525 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:58.991816 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:58.991762 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" event={"ID":"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9","Type":"ContainerStarted","Data":"9b02a172330f9139a6fab77bfca9390050477bdbb3e3e05a619c894a6394f0d6"} Jul 18 02:15:59.963531 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:59.963410 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:15:59.964360 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:59.964324 2548 patch_prober.go:28] interesting pod/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 18 02:15:59.964513 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:15:59.964389 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" podUID="f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 18 02:16:00.949067 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:00.949027 2548 patch_prober.go:28] interesting pod/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 18 02:16:00.949560 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:00.949096 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" podUID="f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 18 02:16:01.954330 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:01.954293 2548 patch_prober.go:28] interesting pod/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 18 02:16:01.954957 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:01.954921 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" podUID="f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 18 02:16:02.949263 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:02.949225 2548 patch_prober.go:28] interesting pod/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 18 02:16:02.949562 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:02.949534 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" podUID="f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 18 02:16:03.953101 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:03.953071 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:16:03.953493 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:03.953359 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:16:03.954165 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:03.954146 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:16:23.580432 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:23.580401 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf"] Jul 18 02:16:23.580992 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:23.580620 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" podUID="f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" containerName="istio-proxy" containerID="cri-o://9b02a172330f9139a6fab77bfca9390050477bdbb3e3e05a619c894a6394f0d6" gracePeriod=30 Jul 18 02:16:28.842206 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.842145 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:16:28.924868 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.924840 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-token\") pod \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " Jul 18 02:16:28.925030 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.924896 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istiod-ca-cert\") pod \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " Jul 18 02:16:28.925030 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.924940 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-workload-socket\") pod \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " Jul 18 02:16:28.925030 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.924977 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-envoy\") pod \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " Jul 18 02:16:28.925030 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.925017 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-workload-certs\") pod \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " Jul 18 02:16:28.925251 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.925046 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-data\") pod \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " Jul 18 02:16:28.925251 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.925065 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-credential-socket\") pod \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " Jul 18 02:16:28.925251 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.925104 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-grsjk\" (UniqueName: \"kubernetes.io/projected/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-kube-api-access-grsjk\") pod \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " Jul 18 02:16:28.925251 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.925128 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-podinfo\") pod \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\" (UID: \"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9\") " Jul 18 02:16:28.925469 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.925241 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" (UID: "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:16:28.925469 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.925419 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" (UID: "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:16:28.925591 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.925466 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-workload-socket\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:16:28.925591 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.925565 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" (UID: "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:16:28.925701 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.925659 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" (UID: "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:16:28.926169 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.926146 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-data" (OuterVolumeSpecName: "istio-data") pod "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" (UID: "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:16:28.927139 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.927112 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" (UID: "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:16:28.927283 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.927263 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-kube-api-access-grsjk" (OuterVolumeSpecName: "kube-api-access-grsjk") pod "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" (UID: "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9"). InnerVolumeSpecName "kube-api-access-grsjk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:16:28.927446 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.927426 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-token" (OuterVolumeSpecName: "istio-token") pod "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" (UID: "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:16:28.927544 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:28.927527 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" (UID: "f4446c38-3abf-41a4-8e81-c9bcfa02ffc9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:16:29.026843 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.026818 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istiod-ca-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:16:29.026843 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.026839 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-envoy\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:16:29.026843 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.026849 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-workload-certs\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:16:29.027022 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.026857 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-data\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:16:29.027022 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.026865 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-credential-socket\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:16:29.027022 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.026873 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-grsjk\" (UniqueName: \"kubernetes.io/projected/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-kube-api-access-grsjk\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:16:29.027022 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.026882 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-podinfo\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:16:29.027022 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.026889 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9-istio-token\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:16:29.156800 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.156750 2548 generic.go:356] "Generic (PLEG): container finished" podID="f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" containerID="9b02a172330f9139a6fab77bfca9390050477bdbb3e3e05a619c894a6394f0d6" exitCode=0 Jul 18 02:16:29.156941 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.156819 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" Jul 18 02:16:29.156941 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.156843 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" event={"ID":"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9","Type":"ContainerDied","Data":"9b02a172330f9139a6fab77bfca9390050477bdbb3e3e05a619c894a6394f0d6"} Jul 18 02:16:29.156941 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.156909 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf" event={"ID":"f4446c38-3abf-41a4-8e81-c9bcfa02ffc9","Type":"ContainerDied","Data":"7b07ab0b0ea58d2dfbbb0ed57600d99893c99800503b0099f2f01e83a3fa04e8"} Jul 18 02:16:29.157102 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.156941 2548 scope.go:117] "RemoveContainer" containerID="9b02a172330f9139a6fab77bfca9390050477bdbb3e3e05a619c894a6394f0d6" Jul 18 02:16:29.166169 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.166154 2548 scope.go:117] "RemoveContainer" containerID="0ce59ab2215ca2336588fb28aaeb958f751d22ab2eda3d91bd799fdb1d804639" Jul 18 02:16:29.174324 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.174308 2548 scope.go:117] "RemoveContainer" containerID="9b02a172330f9139a6fab77bfca9390050477bdbb3e3e05a619c894a6394f0d6" Jul 18 02:16:29.174564 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:16:29.174544 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b02a172330f9139a6fab77bfca9390050477bdbb3e3e05a619c894a6394f0d6\": container with ID starting with 9b02a172330f9139a6fab77bfca9390050477bdbb3e3e05a619c894a6394f0d6 not found: ID does not exist" containerID="9b02a172330f9139a6fab77bfca9390050477bdbb3e3e05a619c894a6394f0d6" Jul 18 02:16:29.174613 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.174571 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b02a172330f9139a6fab77bfca9390050477bdbb3e3e05a619c894a6394f0d6"} err="failed to get container status \"9b02a172330f9139a6fab77bfca9390050477bdbb3e3e05a619c894a6394f0d6\": rpc error: code = NotFound desc = could not find container \"9b02a172330f9139a6fab77bfca9390050477bdbb3e3e05a619c894a6394f0d6\": container with ID starting with 9b02a172330f9139a6fab77bfca9390050477bdbb3e3e05a619c894a6394f0d6 not found: ID does not exist" Jul 18 02:16:29.174613 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.174588 2548 scope.go:117] "RemoveContainer" containerID="0ce59ab2215ca2336588fb28aaeb958f751d22ab2eda3d91bd799fdb1d804639" Jul 18 02:16:29.174822 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:16:29.174803 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ce59ab2215ca2336588fb28aaeb958f751d22ab2eda3d91bd799fdb1d804639\": container with ID starting with 0ce59ab2215ca2336588fb28aaeb958f751d22ab2eda3d91bd799fdb1d804639 not found: ID does not exist" containerID="0ce59ab2215ca2336588fb28aaeb958f751d22ab2eda3d91bd799fdb1d804639" Jul 18 02:16:29.174915 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.174824 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ce59ab2215ca2336588fb28aaeb958f751d22ab2eda3d91bd799fdb1d804639"} err="failed to get container status \"0ce59ab2215ca2336588fb28aaeb958f751d22ab2eda3d91bd799fdb1d804639\": rpc error: code = NotFound desc = could not find container \"0ce59ab2215ca2336588fb28aaeb958f751d22ab2eda3d91bd799fdb1d804639\": container with ID starting with 0ce59ab2215ca2336588fb28aaeb958f751d22ab2eda3d91bd799fdb1d804639 not found: ID does not exist" Jul 18 02:16:29.178466 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.178445 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf"] Jul 18 02:16:29.181917 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.181897 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-23b07985-gw-openshift-default-55b48fc78c-b8fvf"] Jul 18 02:16:29.549711 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:16:29.549623 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" path="/var/lib/kubelet/pods/f4446c38-3abf-41a4-8e81-c9bcfa02ffc9/volumes" Jul 18 02:17:09.203403 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.203369 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5"] Jul 18 02:17:09.204038 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.204018 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" containerName="istio-proxy" Jul 18 02:17:09.204121 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.204041 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" containerName="istio-proxy" Jul 18 02:17:09.204121 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.204083 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" containerName="istio-proxy" Jul 18 02:17:09.204121 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.204090 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" containerName="istio-proxy" Jul 18 02:17:09.204249 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.204164 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" containerName="istio-proxy" Jul 18 02:17:09.204249 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.204172 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4446c38-3abf-41a4-8e81-c9bcfa02ffc9" containerName="istio-proxy" Jul 18 02:17:09.207374 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.207354 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.210303 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.210284 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-05505cc1-openshift-default-dockercfg-fr6bw\"" Jul 18 02:17:09.220134 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.220111 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5"] Jul 18 02:17:09.388282 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.388246 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-credential-socket\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.388282 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.388285 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-token\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.388551 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.388301 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4adcc89-6470-44c5-bb55-76d8f08e0656-istiod-ca-cert\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.388551 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.388425 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-envoy\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.388551 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.388465 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-podinfo\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.388551 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.388494 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp9fj\" (UniqueName: \"kubernetes.io/projected/b4adcc89-6470-44c5-bb55-76d8f08e0656-kube-api-access-kp9fj\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.388719 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.388548 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-workload-certs\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.388719 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.388578 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-workload-socket\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.388719 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.388605 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-data\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.489948 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.489844 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-credential-socket\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.489948 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.489894 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-token\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.489948 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.489916 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4adcc89-6470-44c5-bb55-76d8f08e0656-istiod-ca-cert\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.490261 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.489962 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-envoy\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.490261 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.489993 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-podinfo\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.490261 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.490020 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp9fj\" (UniqueName: \"kubernetes.io/projected/b4adcc89-6470-44c5-bb55-76d8f08e0656-kube-api-access-kp9fj\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.490261 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.490063 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-workload-certs\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.490261 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.490095 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-workload-socket\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.490261 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.490131 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-data\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.490656 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.490356 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-credential-socket\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.490656 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.490449 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-data\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.490656 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.490620 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-workload-socket\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.490988 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.490965 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4adcc89-6470-44c5-bb55-76d8f08e0656-istiod-ca-cert\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.490988 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.490977 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-workload-certs\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.492552 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.492526 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-envoy\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.492723 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.492705 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-podinfo\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.498886 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.498865 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-token\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.498995 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.498888 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp9fj\" (UniqueName: \"kubernetes.io/projected/b4adcc89-6470-44c5-bb55-76d8f08e0656-kube-api-access-kp9fj\") pod \"e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.517850 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.517826 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:09.664672 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.664649 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5"] Jul 18 02:17:09.666591 ip-10-0-137-108 kubenswrapper[2548]: W0718 02:17:09.666561 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4adcc89_6470_44c5_bb55_76d8f08e0656.slice/crio-8c072c15f854aa09c9fe991c829a91346c4c82fd73ee1859804823f3b3a9ec33 WatchSource:0}: Error finding container 8c072c15f854aa09c9fe991c829a91346c4c82fd73ee1859804823f3b3a9ec33: Status 404 returned error can't find the container with id 8c072c15f854aa09c9fe991c829a91346c4c82fd73ee1859804823f3b3a9ec33 Jul 18 02:17:09.668713 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.668679 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:17:09.668838 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.668758 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:17:09.668838 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:09.668815 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:17:10.368139 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:10.368103 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" event={"ID":"b4adcc89-6470-44c5-bb55-76d8f08e0656","Type":"ContainerStarted","Data":"59ecff7921a62f25f007adee8a84f8d9097233cb33cbbae11d7f30e72495d7af"} Jul 18 02:17:10.368541 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:10.368151 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" event={"ID":"b4adcc89-6470-44c5-bb55-76d8f08e0656","Type":"ContainerStarted","Data":"8c072c15f854aa09c9fe991c829a91346c4c82fd73ee1859804823f3b3a9ec33"} Jul 18 02:17:10.388073 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:10.388023 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" podStartSLOduration=1.388007097 podStartE2EDuration="1.388007097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:17:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:17:10.386896611 +0000 UTC m=+1739.239639957" watchObservedRunningTime="2026-07-18 02:17:10.388007097 +0000 UTC m=+1739.240750441" Jul 18 02:17:10.518333 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:10.518300 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:10.523196 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:10.523173 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:11.379107 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:11.379063 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:11.380270 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:11.380252 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:26.380940 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:26.380901 2548 patch_prober.go:28] interesting pod/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 18 02:17:26.381334 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:26.380963 2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" podUID="b4adcc89-6470-44c5-bb55-76d8f08e0656" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 18 02:17:26.459684 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:26.459652 2548 generic.go:356] "Generic (PLEG): container finished" podID="b4adcc89-6470-44c5-bb55-76d8f08e0656" containerID="59ecff7921a62f25f007adee8a84f8d9097233cb33cbbae11d7f30e72495d7af" exitCode=0 Jul 18 02:17:26.459885 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:26.459711 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" event={"ID":"b4adcc89-6470-44c5-bb55-76d8f08e0656","Type":"ContainerDied","Data":"59ecff7921a62f25f007adee8a84f8d9097233cb33cbbae11d7f30e72495d7af"} Jul 18 02:17:26.460171 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:26.460157 2548 scope.go:117] "RemoveContainer" containerID="59ecff7921a62f25f007adee8a84f8d9097233cb33cbbae11d7f30e72495d7af" Jul 18 02:17:26.461141 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:26.461116 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:17:26.461199 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:26.461181 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:17:26.461237 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:26.461216 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:17:27.469793 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:27.469732 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" event={"ID":"b4adcc89-6470-44c5-bb55-76d8f08e0656","Type":"ContainerStarted","Data":"2127b93d49bb6307680cc78f6fa200277d17bc434cf5a90b25da13d52d447304"} Jul 18 02:17:27.518707 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:27.518673 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:27.518930 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:27.518907 2548 patch_prober.go:28] interesting pod/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 18 02:17:27.518984 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:27.518965 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" podUID="b4adcc89-6470-44c5-bb55-76d8f08e0656" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 18 02:17:28.519105 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:28.519068 2548 patch_prober.go:28] interesting pod/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 18 02:17:28.519493 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:28.519133 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" podUID="b4adcc89-6470-44c5-bb55-76d8f08e0656" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 18 02:17:29.518652 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:29.518620 2548 patch_prober.go:28] interesting pod/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 18 02:17:29.518844 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:29.518673 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" podUID="b4adcc89-6470-44c5-bb55-76d8f08e0656" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 18 02:17:30.519085 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:30.519053 2548 patch_prober.go:28] interesting pod/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 18 02:17:30.519471 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:30.519109 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" podUID="b4adcc89-6470-44c5-bb55-76d8f08e0656" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 18 02:17:31.522009 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:31.521983 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:31.522469 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:31.522375 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:31.523132 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:31.523113 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:51.761894 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:51.761753 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5"] Jul 18 02:17:51.762362 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:51.762036 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" podUID="b4adcc89-6470-44c5-bb55-76d8f08e0656" containerName="istio-proxy" containerID="cri-o://2127b93d49bb6307680cc78f6fa200277d17bc434cf5a90b25da13d52d447304" gracePeriod=30 Jul 18 02:17:57.001524 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.001499 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:57.127466 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.127383 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-podinfo\") pod \"b4adcc89-6470-44c5-bb55-76d8f08e0656\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " Jul 18 02:17:57.127466 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.127430 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4adcc89-6470-44c5-bb55-76d8f08e0656-istiod-ca-cert\") pod \"b4adcc89-6470-44c5-bb55-76d8f08e0656\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " Jul 18 02:17:57.127466 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.127457 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-workload-certs\") pod \"b4adcc89-6470-44c5-bb55-76d8f08e0656\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " Jul 18 02:17:57.127690 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.127482 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kp9fj\" (UniqueName: \"kubernetes.io/projected/b4adcc89-6470-44c5-bb55-76d8f08e0656-kube-api-access-kp9fj\") pod \"b4adcc89-6470-44c5-bb55-76d8f08e0656\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " Jul 18 02:17:57.127690 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.127512 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-envoy\") pod \"b4adcc89-6470-44c5-bb55-76d8f08e0656\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " Jul 18 02:17:57.127827 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.127697 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-credential-socket\") pod \"b4adcc89-6470-44c5-bb55-76d8f08e0656\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " Jul 18 02:17:57.127827 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.127735 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b4adcc89-6470-44c5-bb55-76d8f08e0656" (UID: "b4adcc89-6470-44c5-bb55-76d8f08e0656"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:17:57.127827 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.127769 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-token\") pod \"b4adcc89-6470-44c5-bb55-76d8f08e0656\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " Jul 18 02:17:57.127827 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.127820 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-data\") pod \"b4adcc89-6470-44c5-bb55-76d8f08e0656\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " Jul 18 02:17:57.128025 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.127844 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-workload-socket\") pod \"b4adcc89-6470-44c5-bb55-76d8f08e0656\" (UID: \"b4adcc89-6470-44c5-bb55-76d8f08e0656\") " Jul 18 02:17:57.128169 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.127841 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4adcc89-6470-44c5-bb55-76d8f08e0656-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b4adcc89-6470-44c5-bb55-76d8f08e0656" (UID: "b4adcc89-6470-44c5-bb55-76d8f08e0656"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:17:57.128227 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.128164 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-workload-certs\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:17:57.128227 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.127994 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b4adcc89-6470-44c5-bb55-76d8f08e0656" (UID: "b4adcc89-6470-44c5-bb55-76d8f08e0656"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:17:57.128227 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.128212 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b4adcc89-6470-44c5-bb55-76d8f08e0656" (UID: "b4adcc89-6470-44c5-bb55-76d8f08e0656"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:17:57.129080 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.129042 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-data" (OuterVolumeSpecName: "istio-data") pod "b4adcc89-6470-44c5-bb55-76d8f08e0656" (UID: "b4adcc89-6470-44c5-bb55-76d8f08e0656"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:17:57.129722 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.129699 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b4adcc89-6470-44c5-bb55-76d8f08e0656" (UID: "b4adcc89-6470-44c5-bb55-76d8f08e0656"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:17:57.129886 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.129863 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4adcc89-6470-44c5-bb55-76d8f08e0656-kube-api-access-kp9fj" (OuterVolumeSpecName: "kube-api-access-kp9fj") pod "b4adcc89-6470-44c5-bb55-76d8f08e0656" (UID: "b4adcc89-6470-44c5-bb55-76d8f08e0656"). InnerVolumeSpecName "kube-api-access-kp9fj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:17:57.129993 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.129977 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-token" (OuterVolumeSpecName: "istio-token") pod "b4adcc89-6470-44c5-bb55-76d8f08e0656" (UID: "b4adcc89-6470-44c5-bb55-76d8f08e0656"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:17:57.130031 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.130010 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b4adcc89-6470-44c5-bb55-76d8f08e0656" (UID: "b4adcc89-6470-44c5-bb55-76d8f08e0656"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:17:57.229198 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.229173 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-podinfo\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:17:57.229198 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.229197 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b4adcc89-6470-44c5-bb55-76d8f08e0656-istiod-ca-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:17:57.229364 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.229207 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp9fj\" (UniqueName: \"kubernetes.io/projected/b4adcc89-6470-44c5-bb55-76d8f08e0656-kube-api-access-kp9fj\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:17:57.229364 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.229216 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-envoy\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:17:57.229364 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.229226 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-credential-socket\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:17:57.229364 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.229234 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-token\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:17:57.229364 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.229242 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-istio-data\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:17:57.229364 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.229251 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b4adcc89-6470-44c5-bb55-76d8f08e0656-workload-socket\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:17:57.634919 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.634893 2548 generic.go:356] "Generic (PLEG): container finished" podID="b4adcc89-6470-44c5-bb55-76d8f08e0656" containerID="2127b93d49bb6307680cc78f6fa200277d17bc434cf5a90b25da13d52d447304" exitCode=0 Jul 18 02:17:57.635053 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.634945 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" Jul 18 02:17:57.635053 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.634979 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" event={"ID":"b4adcc89-6470-44c5-bb55-76d8f08e0656","Type":"ContainerDied","Data":"2127b93d49bb6307680cc78f6fa200277d17bc434cf5a90b25da13d52d447304"} Jul 18 02:17:57.635053 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.635039 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5" event={"ID":"b4adcc89-6470-44c5-bb55-76d8f08e0656","Type":"ContainerDied","Data":"8c072c15f854aa09c9fe991c829a91346c4c82fd73ee1859804823f3b3a9ec33"} Jul 18 02:17:57.635165 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.635064 2548 scope.go:117] "RemoveContainer" containerID="2127b93d49bb6307680cc78f6fa200277d17bc434cf5a90b25da13d52d447304" Jul 18 02:17:57.644903 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.644884 2548 scope.go:117] "RemoveContainer" containerID="59ecff7921a62f25f007adee8a84f8d9097233cb33cbbae11d7f30e72495d7af" Jul 18 02:17:57.653391 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.653370 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5"] Jul 18 02:17:57.653822 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.653793 2548 scope.go:117] "RemoveContainer" containerID="2127b93d49bb6307680cc78f6fa200277d17bc434cf5a90b25da13d52d447304" Jul 18 02:17:57.654079 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:17:57.654060 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2127b93d49bb6307680cc78f6fa200277d17bc434cf5a90b25da13d52d447304\": container with ID starting with 2127b93d49bb6307680cc78f6fa200277d17bc434cf5a90b25da13d52d447304 not found: ID does not exist" containerID="2127b93d49bb6307680cc78f6fa200277d17bc434cf5a90b25da13d52d447304" Jul 18 02:17:57.654195 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.654089 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2127b93d49bb6307680cc78f6fa200277d17bc434cf5a90b25da13d52d447304"} err="failed to get container status \"2127b93d49bb6307680cc78f6fa200277d17bc434cf5a90b25da13d52d447304\": rpc error: code = NotFound desc = could not find container \"2127b93d49bb6307680cc78f6fa200277d17bc434cf5a90b25da13d52d447304\": container with ID starting with 2127b93d49bb6307680cc78f6fa200277d17bc434cf5a90b25da13d52d447304 not found: ID does not exist" Jul 18 02:17:57.654195 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.654113 2548 scope.go:117] "RemoveContainer" containerID="59ecff7921a62f25f007adee8a84f8d9097233cb33cbbae11d7f30e72495d7af" Jul 18 02:17:57.654502 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:17:57.654476 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59ecff7921a62f25f007adee8a84f8d9097233cb33cbbae11d7f30e72495d7af\": container with ID starting with 59ecff7921a62f25f007adee8a84f8d9097233cb33cbbae11d7f30e72495d7af not found: ID does not exist" containerID="59ecff7921a62f25f007adee8a84f8d9097233cb33cbbae11d7f30e72495d7af" Jul 18 02:17:57.654611 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.654510 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59ecff7921a62f25f007adee8a84f8d9097233cb33cbbae11d7f30e72495d7af"} err="failed to get container status \"59ecff7921a62f25f007adee8a84f8d9097233cb33cbbae11d7f30e72495d7af\": rpc error: code = NotFound desc = could not find container \"59ecff7921a62f25f007adee8a84f8d9097233cb33cbbae11d7f30e72495d7af\": container with ID starting with 59ecff7921a62f25f007adee8a84f8d9097233cb33cbbae11d7f30e72495d7af not found: ID does not exist" Jul 18 02:17:57.656608 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:57.656589 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-05505cc1-openshift-default-794bf64655-vjqw5"] Jul 18 02:17:59.551534 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:17:59.551505 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4adcc89-6470-44c5-bb55-76d8f08e0656" path="/var/lib/kubelet/pods/b4adcc89-6470-44c5-bb55-76d8f08e0656/volumes" Jul 18 02:18:11.646181 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:18:11.646154 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 02:18:11.664247 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:18:11.664226 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 02:19:30.616990 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.616956 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w"] Jul 18 02:19:30.617582 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.617555 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4adcc89-6470-44c5-bb55-76d8f08e0656" containerName="istio-proxy" Jul 18 02:19:30.617582 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.617574 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4adcc89-6470-44c5-bb55-76d8f08e0656" containerName="istio-proxy" Jul 18 02:19:30.617703 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.617593 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4adcc89-6470-44c5-bb55-76d8f08e0656" containerName="istio-proxy" Jul 18 02:19:30.617703 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.617602 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4adcc89-6470-44c5-bb55-76d8f08e0656" containerName="istio-proxy" Jul 18 02:19:30.617835 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.617714 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4adcc89-6470-44c5-bb55-76d8f08e0656" containerName="istio-proxy" Jul 18 02:19:30.617835 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.617730 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4adcc89-6470-44c5-bb55-76d8f08e0656" containerName="istio-proxy" Jul 18 02:19:30.621003 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.620982 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.623794 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.623762 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-b9dbc17d-openshift-default-dockercfg-pr6mw\"" Jul 18 02:19:30.643698 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.643674 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w"] Jul 18 02:19:30.734801 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.734743 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-workload-certs\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.734953 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.734810 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw54c\" (UniqueName: \"kubernetes.io/projected/a4122dc2-0047-4041-a38c-29eddb36084f-kube-api-access-cw54c\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.734953 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.734864 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-workload-socket\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.734953 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.734916 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-credential-socket\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.734953 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.734936 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4122dc2-0047-4041-a38c-29eddb36084f-istiod-ca-cert\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.734953 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.734954 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4122dc2-0047-4041-a38c-29eddb36084f-istio-podinfo\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.735121 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.734997 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4122dc2-0047-4041-a38c-29eddb36084f-istio-token\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.735121 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.735056 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-istio-envoy\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.735121 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.735073 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-istio-data\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.836520 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.836490 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-workload-certs\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.836687 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.836540 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw54c\" (UniqueName: \"kubernetes.io/projected/a4122dc2-0047-4041-a38c-29eddb36084f-kube-api-access-cw54c\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.836687 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.836564 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-workload-socket\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.836687 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.836597 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-credential-socket\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.836687 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.836621 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4122dc2-0047-4041-a38c-29eddb36084f-istiod-ca-cert\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.836687 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.836641 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4122dc2-0047-4041-a38c-29eddb36084f-istio-podinfo\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.837000 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.836805 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4122dc2-0047-4041-a38c-29eddb36084f-istio-token\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.837000 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.836904 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-istio-envoy\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.837000 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.836931 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-istio-data\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.837000 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.836990 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-workload-socket\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.837214 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.836931 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-workload-certs\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.837214 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.837031 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-credential-socket\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.837309 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.837225 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-istio-data\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.837392 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.837373 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4122dc2-0047-4041-a38c-29eddb36084f-istiod-ca-cert\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.839168 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.839139 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-istio-envoy\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.839306 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.839289 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4122dc2-0047-4041-a38c-29eddb36084f-istio-podinfo\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.844830 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.844770 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4122dc2-0047-4041-a38c-29eddb36084f-istio-token\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.845120 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.845099 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw54c\" (UniqueName: \"kubernetes.io/projected/a4122dc2-0047-4041-a38c-29eddb36084f-kube-api-access-cw54c\") pod \"e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:30.931404 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:30.931378 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:31.075181 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:31.075151 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w"] Jul 18 02:19:31.076514 ip-10-0-137-108 kubenswrapper[2548]: W0718 02:19:31.076491 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4122dc2_0047_4041_a38c_29eddb36084f.slice/crio-bcd52469c003ec120acb3bd1b2fa4591a14a5f35e58b0eec502063309b0a4e5e WatchSource:0}: Error finding container bcd52469c003ec120acb3bd1b2fa4591a14a5f35e58b0eec502063309b0a4e5e: Status 404 returned error can't find the container with id bcd52469c003ec120acb3bd1b2fa4591a14a5f35e58b0eec502063309b0a4e5e Jul 18 02:19:31.078896 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:31.078854 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:19:31.078998 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:31.078942 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:19:31.078998 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:31.078989 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:19:31.121759 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:31.121739 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" event={"ID":"a4122dc2-0047-4041-a38c-29eddb36084f","Type":"ContainerStarted","Data":"bcd52469c003ec120acb3bd1b2fa4591a14a5f35e58b0eec502063309b0a4e5e"} Jul 18 02:19:32.129669 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:32.129630 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" event={"ID":"a4122dc2-0047-4041-a38c-29eddb36084f","Type":"ContainerStarted","Data":"ffbb191838a55e2a7818c3f612f7e4a214e5fd2dc6265094815d57e86a59483d"} Jul 18 02:19:32.155347 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:32.155281 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" podStartSLOduration=2.155264948 podStartE2EDuration="2.155264948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:19:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:19:32.153635414 +0000 UTC m=+1881.006378759" watchObservedRunningTime="2026-07-18 02:19:32.155264948 +0000 UTC m=+1881.008008292" Jul 18 02:19:32.932003 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:32.931971 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:32.936469 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:32.936446 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:33.134571 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:33.134532 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:33.135328 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:33.135311 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:47.216047 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:47.215961 2548 generic.go:356] "Generic (PLEG): container finished" podID="a4122dc2-0047-4041-a38c-29eddb36084f" containerID="ffbb191838a55e2a7818c3f612f7e4a214e5fd2dc6265094815d57e86a59483d" exitCode=0 Jul 18 02:19:47.216047 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:47.216033 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" event={"ID":"a4122dc2-0047-4041-a38c-29eddb36084f","Type":"ContainerDied","Data":"ffbb191838a55e2a7818c3f612f7e4a214e5fd2dc6265094815d57e86a59483d"} Jul 18 02:19:47.216469 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:47.216430 2548 scope.go:117] "RemoveContainer" containerID="ffbb191838a55e2a7818c3f612f7e4a214e5fd2dc6265094815d57e86a59483d" Jul 18 02:19:47.217459 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:47.217434 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:19:47.217516 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:47.217499 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:19:47.217552 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:47.217531 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:19:48.134495 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:48.134462 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:48.224023 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:48.223985 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" event={"ID":"a4122dc2-0047-4041-a38c-29eddb36084f","Type":"ContainerStarted","Data":"abde7e5689a185286cc34c22f64422721bc3778938618e562949a6d24a8ebfa4"} Jul 18 02:19:48.932277 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:48.932246 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:48.933470 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:48.933438 2548 patch_prober.go:28] interesting pod/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 18 02:19:48.933588 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:48.933499 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" podUID="a4122dc2-0047-4041-a38c-29eddb36084f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 18 02:19:49.931967 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:49.931934 2548 patch_prober.go:28] interesting pod/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 18 02:19:49.932327 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:49.931988 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" podUID="a4122dc2-0047-4041-a38c-29eddb36084f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 18 02:19:50.931997 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:50.931967 2548 patch_prober.go:28] interesting pod/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 18 02:19:50.932381 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:50.932015 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" podUID="a4122dc2-0047-4041-a38c-29eddb36084f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 18 02:19:51.951762 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:51.951729 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:51.952182 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:51.951987 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:19:51.952673 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:19:51.952655 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:20:07.404147 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:07.404112 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w"] Jul 18 02:20:07.404642 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:07.404357 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" podUID="a4122dc2-0047-4041-a38c-29eddb36084f" containerName="istio-proxy" containerID="cri-o://abde7e5689a185286cc34c22f64422721bc3778938618e562949a6d24a8ebfa4" gracePeriod=30 Jul 18 02:20:12.651946 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.651923 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:20:12.730253 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.730186 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-istio-envoy\") pod \"a4122dc2-0047-4041-a38c-29eddb36084f\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " Jul 18 02:20:12.730253 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.730233 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cw54c\" (UniqueName: \"kubernetes.io/projected/a4122dc2-0047-4041-a38c-29eddb36084f-kube-api-access-cw54c\") pod \"a4122dc2-0047-4041-a38c-29eddb36084f\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " Jul 18 02:20:12.730418 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.730260 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4122dc2-0047-4041-a38c-29eddb36084f-istiod-ca-cert\") pod \"a4122dc2-0047-4041-a38c-29eddb36084f\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " Jul 18 02:20:12.730460 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.730427 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-credential-socket\") pod \"a4122dc2-0047-4041-a38c-29eddb36084f\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " Jul 18 02:20:12.730507 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.730461 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4122dc2-0047-4041-a38c-29eddb36084f-istio-podinfo\") pod \"a4122dc2-0047-4041-a38c-29eddb36084f\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " Jul 18 02:20:12.730507 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.730478 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4122dc2-0047-4041-a38c-29eddb36084f-istio-token\") pod \"a4122dc2-0047-4041-a38c-29eddb36084f\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " Jul 18 02:20:12.730627 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.730505 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-workload-socket\") pod \"a4122dc2-0047-4041-a38c-29eddb36084f\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " Jul 18 02:20:12.730812 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.730712 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a4122dc2-0047-4041-a38c-29eddb36084f" (UID: "a4122dc2-0047-4041-a38c-29eddb36084f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:20:12.730812 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.730732 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-istio-data\") pod \"a4122dc2-0047-4041-a38c-29eddb36084f\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " Jul 18 02:20:12.730812 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.730728 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4122dc2-0047-4041-a38c-29eddb36084f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a4122dc2-0047-4041-a38c-29eddb36084f" (UID: "a4122dc2-0047-4041-a38c-29eddb36084f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:20:12.730812 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.730804 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-workload-certs\") pod \"a4122dc2-0047-4041-a38c-29eddb36084f\" (UID: \"a4122dc2-0047-4041-a38c-29eddb36084f\") " Jul 18 02:20:12.731075 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.730860 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a4122dc2-0047-4041-a38c-29eddb36084f" (UID: "a4122dc2-0047-4041-a38c-29eddb36084f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:20:12.731075 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.731044 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a4122dc2-0047-4041-a38c-29eddb36084f" (UID: "a4122dc2-0047-4041-a38c-29eddb36084f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:20:12.731208 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.731188 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4122dc2-0047-4041-a38c-29eddb36084f-istiod-ca-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:20:12.731208 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.731212 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-credential-socket\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:20:12.731323 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.731226 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-workload-socket\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:20:12.731323 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.731240 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-workload-certs\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:20:12.731608 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.731591 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-istio-data" (OuterVolumeSpecName: "istio-data") pod "a4122dc2-0047-4041-a38c-29eddb36084f" (UID: "a4122dc2-0047-4041-a38c-29eddb36084f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:20:12.733003 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.732984 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a4122dc2-0047-4041-a38c-29eddb36084f" (UID: "a4122dc2-0047-4041-a38c-29eddb36084f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:20:12.733106 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.733013 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4122dc2-0047-4041-a38c-29eddb36084f-istio-token" (OuterVolumeSpecName: "istio-token") pod "a4122dc2-0047-4041-a38c-29eddb36084f" (UID: "a4122dc2-0047-4041-a38c-29eddb36084f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:20:12.733106 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.733069 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a4122dc2-0047-4041-a38c-29eddb36084f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a4122dc2-0047-4041-a38c-29eddb36084f" (UID: "a4122dc2-0047-4041-a38c-29eddb36084f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:20:12.733106 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.733074 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4122dc2-0047-4041-a38c-29eddb36084f-kube-api-access-cw54c" (OuterVolumeSpecName: "kube-api-access-cw54c") pod "a4122dc2-0047-4041-a38c-29eddb36084f" (UID: "a4122dc2-0047-4041-a38c-29eddb36084f"). InnerVolumeSpecName "kube-api-access-cw54c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:20:12.831834 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.831813 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4122dc2-0047-4041-a38c-29eddb36084f-istio-podinfo\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:20:12.831834 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.831832 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4122dc2-0047-4041-a38c-29eddb36084f-istio-token\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:20:12.831976 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.831842 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-istio-data\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:20:12.831976 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.831849 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4122dc2-0047-4041-a38c-29eddb36084f-istio-envoy\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:20:12.831976 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:12.831858 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cw54c\" (UniqueName: \"kubernetes.io/projected/a4122dc2-0047-4041-a38c-29eddb36084f-kube-api-access-cw54c\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:20:13.356660 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:13.356607 2548 generic.go:356] "Generic (PLEG): container finished" podID="a4122dc2-0047-4041-a38c-29eddb36084f" containerID="abde7e5689a185286cc34c22f64422721bc3778938618e562949a6d24a8ebfa4" exitCode=0 Jul 18 02:20:13.356912 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:13.356668 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" event={"ID":"a4122dc2-0047-4041-a38c-29eddb36084f","Type":"ContainerDied","Data":"abde7e5689a185286cc34c22f64422721bc3778938618e562949a6d24a8ebfa4"} Jul 18 02:20:13.356912 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:13.356748 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" event={"ID":"a4122dc2-0047-4041-a38c-29eddb36084f","Type":"ContainerDied","Data":"bcd52469c003ec120acb3bd1b2fa4591a14a5f35e58b0eec502063309b0a4e5e"} Jul 18 02:20:13.356912 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:13.356690 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w" Jul 18 02:20:13.356912 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:13.356796 2548 scope.go:117] "RemoveContainer" containerID="abde7e5689a185286cc34c22f64422721bc3778938618e562949a6d24a8ebfa4" Jul 18 02:20:13.367036 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:13.367020 2548 scope.go:117] "RemoveContainer" containerID="ffbb191838a55e2a7818c3f612f7e4a214e5fd2dc6265094815d57e86a59483d" Jul 18 02:20:13.375176 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:13.375160 2548 scope.go:117] "RemoveContainer" containerID="abde7e5689a185286cc34c22f64422721bc3778938618e562949a6d24a8ebfa4" Jul 18 02:20:13.375438 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:20:13.375422 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abde7e5689a185286cc34c22f64422721bc3778938618e562949a6d24a8ebfa4\": container with ID starting with abde7e5689a185286cc34c22f64422721bc3778938618e562949a6d24a8ebfa4 not found: ID does not exist" containerID="abde7e5689a185286cc34c22f64422721bc3778938618e562949a6d24a8ebfa4" Jul 18 02:20:13.375492 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:13.375445 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abde7e5689a185286cc34c22f64422721bc3778938618e562949a6d24a8ebfa4"} err="failed to get container status \"abde7e5689a185286cc34c22f64422721bc3778938618e562949a6d24a8ebfa4\": rpc error: code = NotFound desc = could not find container \"abde7e5689a185286cc34c22f64422721bc3778938618e562949a6d24a8ebfa4\": container with ID starting with abde7e5689a185286cc34c22f64422721bc3778938618e562949a6d24a8ebfa4 not found: ID does not exist" Jul 18 02:20:13.375492 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:13.375461 2548 scope.go:117] "RemoveContainer" containerID="ffbb191838a55e2a7818c3f612f7e4a214e5fd2dc6265094815d57e86a59483d" Jul 18 02:20:13.375697 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:20:13.375678 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffbb191838a55e2a7818c3f612f7e4a214e5fd2dc6265094815d57e86a59483d\": container with ID starting with ffbb191838a55e2a7818c3f612f7e4a214e5fd2dc6265094815d57e86a59483d not found: ID does not exist" containerID="ffbb191838a55e2a7818c3f612f7e4a214e5fd2dc6265094815d57e86a59483d" Jul 18 02:20:13.375749 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:13.375705 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffbb191838a55e2a7818c3f612f7e4a214e5fd2dc6265094815d57e86a59483d"} err="failed to get container status \"ffbb191838a55e2a7818c3f612f7e4a214e5fd2dc6265094815d57e86a59483d\": rpc error: code = NotFound desc = could not find container \"ffbb191838a55e2a7818c3f612f7e4a214e5fd2dc6265094815d57e86a59483d\": container with ID starting with ffbb191838a55e2a7818c3f612f7e4a214e5fd2dc6265094815d57e86a59483d not found: ID does not exist" Jul 18 02:20:13.380817 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:13.380792 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w"] Jul 18 02:20:13.385141 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:13.385120 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-b9dbc17d-openshift-default-64b8d59dbc-lls4w"] Jul 18 02:20:13.549134 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:13.549109 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4122dc2-0047-4041-a38c-29eddb36084f" path="/var/lib/kubelet/pods/a4122dc2-0047-4041-a38c-29eddb36084f/volumes" Jul 18 02:20:27.089657 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.089625 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6"] Jul 18 02:20:27.090233 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.090213 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4122dc2-0047-4041-a38c-29eddb36084f" containerName="istio-proxy" Jul 18 02:20:27.090315 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.090236 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4122dc2-0047-4041-a38c-29eddb36084f" containerName="istio-proxy" Jul 18 02:20:27.090381 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.090368 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4122dc2-0047-4041-a38c-29eddb36084f" containerName="istio-proxy" Jul 18 02:20:27.090442 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.090388 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4122dc2-0047-4041-a38c-29eddb36084f" containerName="istio-proxy" Jul 18 02:20:27.093588 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.093569 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.096061 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.096041 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-546eb0de-openshift-default-dockercfg-rs6kq\"" Jul 18 02:20:27.103656 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.103626 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6"] Jul 18 02:20:27.259167 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.259131 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-podinfo\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.259167 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.259177 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-credential-socket\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.259427 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.259192 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c8d9594-f547-4033-915d-6fd8a61ee68e-istiod-ca-cert\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.259427 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.259288 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-workload-socket\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.259427 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.259322 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-envoy\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.259427 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.259343 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-workload-certs\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.259427 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.259367 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-token\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.259427 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.259414 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz8jk\" (UniqueName: \"kubernetes.io/projected/5c8d9594-f547-4033-915d-6fd8a61ee68e-kube-api-access-bz8jk\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.259732 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.259493 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-data\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.360841 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.360754 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz8jk\" (UniqueName: \"kubernetes.io/projected/5c8d9594-f547-4033-915d-6fd8a61ee68e-kube-api-access-bz8jk\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.361004 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.360857 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-data\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.361004 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.360913 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-podinfo\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.361004 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.360956 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-credential-socket\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.361004 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.360980 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c8d9594-f547-4033-915d-6fd8a61ee68e-istiod-ca-cert\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.361205 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.361110 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-workload-socket\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.361205 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.361180 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-envoy\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.361315 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.361212 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-workload-certs\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.361315 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.361241 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-token\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.361315 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.361279 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-data\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.361472 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.361336 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-credential-socket\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.361529 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.361485 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-workload-socket\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.361626 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.361602 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-workload-certs\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.361803 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.361763 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c8d9594-f547-4033-915d-6fd8a61ee68e-istiod-ca-cert\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.363500 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.363476 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-envoy\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.363664 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.363646 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-podinfo\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.368290 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.368268 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-token\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.368499 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.368481 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz8jk\" (UniqueName: \"kubernetes.io/projected/5c8d9594-f547-4033-915d-6fd8a61ee68e-kube-api-access-bz8jk\") pod \"e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.404530 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.404507 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:27.557567 ip-10-0-137-108 kubenswrapper[2548]: W0718 02:20:27.557535 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c8d9594_f547_4033_915d_6fd8a61ee68e.slice/crio-e8f7af7518777998613ce9305d93b9fda7142b59012be634214af4ebcc764a90 WatchSource:0}: Error finding container e8f7af7518777998613ce9305d93b9fda7142b59012be634214af4ebcc764a90: Status 404 returned error can't find the container with id e8f7af7518777998613ce9305d93b9fda7142b59012be634214af4ebcc764a90 Jul 18 02:20:27.558984 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.558958 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6"] Jul 18 02:20:27.559948 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.559916 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:20:27.560049 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.559978 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:20:27.560049 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:27.560006 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:20:28.437175 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:28.437145 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" event={"ID":"5c8d9594-f547-4033-915d-6fd8a61ee68e","Type":"ContainerStarted","Data":"e68ea4d69a99d880011a964b65e52d03a0080b86a438d7ba0a79608130f8732c"} Jul 18 02:20:28.437612 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:28.437195 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" event={"ID":"5c8d9594-f547-4033-915d-6fd8a61ee68e","Type":"ContainerStarted","Data":"e8f7af7518777998613ce9305d93b9fda7142b59012be634214af4ebcc764a90"} Jul 18 02:20:28.460459 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:28.460401 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" podStartSLOduration=1.460385413 podStartE2EDuration="1.460385413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:20:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:20:28.458220345 +0000 UTC m=+1937.310963690" watchObservedRunningTime="2026-07-18 02:20:28.460385413 +0000 UTC m=+1937.313128757" Jul 18 02:20:29.404939 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:29.404906 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:29.409811 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:29.409788 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:29.442313 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:29.442286 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:29.443223 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:29.443203 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:43.517860 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:43.517826 2548 generic.go:356] "Generic (PLEG): container finished" podID="5c8d9594-f547-4033-915d-6fd8a61ee68e" containerID="e68ea4d69a99d880011a964b65e52d03a0080b86a438d7ba0a79608130f8732c" exitCode=0 Jul 18 02:20:43.518250 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:43.517903 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" event={"ID":"5c8d9594-f547-4033-915d-6fd8a61ee68e","Type":"ContainerDied","Data":"e68ea4d69a99d880011a964b65e52d03a0080b86a438d7ba0a79608130f8732c"} Jul 18 02:20:43.518311 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:43.518297 2548 scope.go:117] "RemoveContainer" containerID="e68ea4d69a99d880011a964b65e52d03a0080b86a438d7ba0a79608130f8732c" Jul 18 02:20:43.519328 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:43.519303 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:20:43.519388 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:43.519369 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:20:43.519439 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:43.519400 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:20:44.442728 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:44.442695 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:44.529126 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:44.529089 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" event={"ID":"5c8d9594-f547-4033-915d-6fd8a61ee68e","Type":"ContainerStarted","Data":"aa9b03e2e709220228b9933d3baa4f52d3529dd4984dcdccde9de299c9d7ebe3"} Jul 18 02:20:45.405309 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:45.405271 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:45.406612 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:45.406586 2548 patch_prober.go:28] interesting pod/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 18 02:20:45.406732 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:45.406636 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" podUID="5c8d9594-f547-4033-915d-6fd8a61ee68e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 18 02:20:46.405625 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:46.405585 2548 patch_prober.go:28] interesting pod/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 18 02:20:46.406031 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:46.405649 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" podUID="5c8d9594-f547-4033-915d-6fd8a61ee68e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 18 02:20:47.405663 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:47.405625 2548 patch_prober.go:28] interesting pod/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 18 02:20:47.406130 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:47.405685 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" podUID="5c8d9594-f547-4033-915d-6fd8a61ee68e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 18 02:20:48.405118 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:48.405083 2548 patch_prober.go:28] interesting pod/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 18 02:20:48.405304 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:48.405149 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" podUID="5c8d9594-f547-4033-915d-6fd8a61ee68e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 18 02:20:49.408510 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:49.408480 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:49.408946 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:49.408824 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:20:49.409418 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:20:49.409400 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:22:11.757949 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:11.757915 2548 scope.go:117] "RemoveContainer" containerID="af3bcd6162905ab4f51913e010e555ace7be7ad1389b63694c00b2a91743255f" Jul 18 02:22:13.695298 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:13.695261 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6"] Jul 18 02:22:13.695726 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:13.695551 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" podUID="5c8d9594-f547-4033-915d-6fd8a61ee68e" containerName="istio-proxy" containerID="cri-o://aa9b03e2e709220228b9933d3baa4f52d3529dd4984dcdccde9de299c9d7ebe3" gracePeriod=30 Jul 18 02:22:18.945157 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:18.945131 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:22:19.024559 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.024464 2548 generic.go:356] "Generic (PLEG): container finished" podID="5c8d9594-f547-4033-915d-6fd8a61ee68e" containerID="aa9b03e2e709220228b9933d3baa4f52d3529dd4984dcdccde9de299c9d7ebe3" exitCode=0 Jul 18 02:22:19.024559 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.024511 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" Jul 18 02:22:19.024846 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.024560 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" event={"ID":"5c8d9594-f547-4033-915d-6fd8a61ee68e","Type":"ContainerDied","Data":"aa9b03e2e709220228b9933d3baa4f52d3529dd4984dcdccde9de299c9d7ebe3"} Jul 18 02:22:19.024846 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.024605 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6" event={"ID":"5c8d9594-f547-4033-915d-6fd8a61ee68e","Type":"ContainerDied","Data":"e8f7af7518777998613ce9305d93b9fda7142b59012be634214af4ebcc764a90"} Jul 18 02:22:19.024846 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.024627 2548 scope.go:117] "RemoveContainer" containerID="aa9b03e2e709220228b9933d3baa4f52d3529dd4984dcdccde9de299c9d7ebe3" Jul 18 02:22:19.035168 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.035145 2548 scope.go:117] "RemoveContainer" containerID="e68ea4d69a99d880011a964b65e52d03a0080b86a438d7ba0a79608130f8732c" Jul 18 02:22:19.044287 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.044265 2548 scope.go:117] "RemoveContainer" containerID="aa9b03e2e709220228b9933d3baa4f52d3529dd4984dcdccde9de299c9d7ebe3" Jul 18 02:22:19.044570 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:22:19.044547 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa9b03e2e709220228b9933d3baa4f52d3529dd4984dcdccde9de299c9d7ebe3\": container with ID starting with aa9b03e2e709220228b9933d3baa4f52d3529dd4984dcdccde9de299c9d7ebe3 not found: ID does not exist" containerID="aa9b03e2e709220228b9933d3baa4f52d3529dd4984dcdccde9de299c9d7ebe3" Jul 18 02:22:19.044635 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.044580 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa9b03e2e709220228b9933d3baa4f52d3529dd4984dcdccde9de299c9d7ebe3"} err="failed to get container status \"aa9b03e2e709220228b9933d3baa4f52d3529dd4984dcdccde9de299c9d7ebe3\": rpc error: code = NotFound desc = could not find container \"aa9b03e2e709220228b9933d3baa4f52d3529dd4984dcdccde9de299c9d7ebe3\": container with ID starting with aa9b03e2e709220228b9933d3baa4f52d3529dd4984dcdccde9de299c9d7ebe3 not found: ID does not exist" Jul 18 02:22:19.044635 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.044599 2548 scope.go:117] "RemoveContainer" containerID="e68ea4d69a99d880011a964b65e52d03a0080b86a438d7ba0a79608130f8732c" Jul 18 02:22:19.044878 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:22:19.044860 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e68ea4d69a99d880011a964b65e52d03a0080b86a438d7ba0a79608130f8732c\": container with ID starting with e68ea4d69a99d880011a964b65e52d03a0080b86a438d7ba0a79608130f8732c not found: ID does not exist" containerID="e68ea4d69a99d880011a964b65e52d03a0080b86a438d7ba0a79608130f8732c" Jul 18 02:22:19.044963 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.044888 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e68ea4d69a99d880011a964b65e52d03a0080b86a438d7ba0a79608130f8732c"} err="failed to get container status \"e68ea4d69a99d880011a964b65e52d03a0080b86a438d7ba0a79608130f8732c\": rpc error: code = NotFound desc = could not find container \"e68ea4d69a99d880011a964b65e52d03a0080b86a438d7ba0a79608130f8732c\": container with ID starting with e68ea4d69a99d880011a964b65e52d03a0080b86a438d7ba0a79608130f8732c not found: ID does not exist" Jul 18 02:22:19.117269 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.117234 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c8d9594-f547-4033-915d-6fd8a61ee68e-istiod-ca-cert\") pod \"5c8d9594-f547-4033-915d-6fd8a61ee68e\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " Jul 18 02:22:19.117428 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.117311 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-workload-certs\") pod \"5c8d9594-f547-4033-915d-6fd8a61ee68e\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " Jul 18 02:22:19.117477 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.117420 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-envoy\") pod \"5c8d9594-f547-4033-915d-6fd8a61ee68e\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " Jul 18 02:22:19.117477 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.117464 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-podinfo\") pod \"5c8d9594-f547-4033-915d-6fd8a61ee68e\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " Jul 18 02:22:19.117564 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.117497 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz8jk\" (UniqueName: \"kubernetes.io/projected/5c8d9594-f547-4033-915d-6fd8a61ee68e-kube-api-access-bz8jk\") pod \"5c8d9594-f547-4033-915d-6fd8a61ee68e\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " Jul 18 02:22:19.117564 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.117540 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5c8d9594-f547-4033-915d-6fd8a61ee68e" (UID: "5c8d9594-f547-4033-915d-6fd8a61ee68e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:22:19.117673 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.117578 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-workload-socket\") pod \"5c8d9594-f547-4033-915d-6fd8a61ee68e\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " Jul 18 02:22:19.117673 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.117603 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-data\") pod \"5c8d9594-f547-4033-915d-6fd8a61ee68e\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " Jul 18 02:22:19.117673 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.117628 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c8d9594-f547-4033-915d-6fd8a61ee68e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5c8d9594-f547-4033-915d-6fd8a61ee68e" (UID: "5c8d9594-f547-4033-915d-6fd8a61ee68e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:22:19.117673 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.117653 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-credential-socket\") pod \"5c8d9594-f547-4033-915d-6fd8a61ee68e\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " Jul 18 02:22:19.117925 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.117685 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-token\") pod \"5c8d9594-f547-4033-915d-6fd8a61ee68e\" (UID: \"5c8d9594-f547-4033-915d-6fd8a61ee68e\") " Jul 18 02:22:19.117925 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.117831 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5c8d9594-f547-4033-915d-6fd8a61ee68e" (UID: "5c8d9594-f547-4033-915d-6fd8a61ee68e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:22:19.118051 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.117966 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5c8d9594-f547-4033-915d-6fd8a61ee68e" (UID: "5c8d9594-f547-4033-915d-6fd8a61ee68e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:22:19.118109 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.118050 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c8d9594-f547-4033-915d-6fd8a61ee68e-istiod-ca-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:22:19.118109 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.118072 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-workload-certs\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:22:19.118109 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.118085 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-workload-socket\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:22:19.118109 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.118099 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-credential-socket\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:22:19.118499 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.118462 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-data" (OuterVolumeSpecName: "istio-data") pod "5c8d9594-f547-4033-915d-6fd8a61ee68e" (UID: "5c8d9594-f547-4033-915d-6fd8a61ee68e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:22:19.119868 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.119840 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5c8d9594-f547-4033-915d-6fd8a61ee68e" (UID: "5c8d9594-f547-4033-915d-6fd8a61ee68e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:22:19.119868 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.119862 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c8d9594-f547-4033-915d-6fd8a61ee68e-kube-api-access-bz8jk" (OuterVolumeSpecName: "kube-api-access-bz8jk") pod "5c8d9594-f547-4033-915d-6fd8a61ee68e" (UID: "5c8d9594-f547-4033-915d-6fd8a61ee68e"). InnerVolumeSpecName "kube-api-access-bz8jk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:22:19.120169 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.120155 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5c8d9594-f547-4033-915d-6fd8a61ee68e" (UID: "5c8d9594-f547-4033-915d-6fd8a61ee68e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:22:19.120337 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.120323 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-token" (OuterVolumeSpecName: "istio-token") pod "5c8d9594-f547-4033-915d-6fd8a61ee68e" (UID: "5c8d9594-f547-4033-915d-6fd8a61ee68e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:22:19.219540 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.219507 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-envoy\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:22:19.219540 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.219537 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-podinfo\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:22:19.219540 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.219547 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz8jk\" (UniqueName: \"kubernetes.io/projected/5c8d9594-f547-4033-915d-6fd8a61ee68e-kube-api-access-bz8jk\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:22:19.219812 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.219556 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-data\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:22:19.219812 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.219564 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c8d9594-f547-4033-915d-6fd8a61ee68e-istio-token\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:22:19.351911 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.351878 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6"] Jul 18 02:22:19.356082 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.356054 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-546eb0de-openshift-default-786f7fd98c-r9dx6"] Jul 18 02:22:19.558072 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:22:19.558039 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c8d9594-f547-4033-915d-6fd8a61ee68e" path="/var/lib/kubelet/pods/5c8d9594-f547-4033-915d-6fd8a61ee68e/volumes" Jul 18 02:23:10.256163 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.256126 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95"] Jul 18 02:23:10.256629 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.256525 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c8d9594-f547-4033-915d-6fd8a61ee68e" containerName="istio-proxy" Jul 18 02:23:10.256629 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.256535 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c8d9594-f547-4033-915d-6fd8a61ee68e" containerName="istio-proxy" Jul 18 02:23:10.256629 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.256599 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4122dc2-0047-4041-a38c-29eddb36084f" containerName="istio-proxy" Jul 18 02:23:10.256629 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.256609 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4122dc2-0047-4041-a38c-29eddb36084f" containerName="istio-proxy" Jul 18 02:23:10.256629 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.256627 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c8d9594-f547-4033-915d-6fd8a61ee68e" containerName="istio-proxy" Jul 18 02:23:10.256838 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.256637 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c8d9594-f547-4033-915d-6fd8a61ee68e" containerName="istio-proxy" Jul 18 02:23:10.256838 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.256724 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c8d9594-f547-4033-915d-6fd8a61ee68e" containerName="istio-proxy" Jul 18 02:23:10.260441 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.260420 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.262904 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.262886 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-8b6a54-openshift-default-dockercfg-snvwl\"" Jul 18 02:23:10.273383 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.273357 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95"] Jul 18 02:23:10.380103 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.380063 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-workload-certs\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.380270 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.380116 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d3df20a-43d3-437b-a707-616076c7455e-istiod-ca-cert\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.380270 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.380196 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-istio-envoy\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.380270 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.380263 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-credential-socket\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.380375 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.380305 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d3df20a-43d3-437b-a707-616076c7455e-istio-podinfo\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.380375 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.380368 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-workload-socket\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.380441 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.380422 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx4br\" (UniqueName: \"kubernetes.io/projected/5d3df20a-43d3-437b-a707-616076c7455e-kube-api-access-lx4br\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.380475 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.380453 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d3df20a-43d3-437b-a707-616076c7455e-istio-token\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.380531 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.380512 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-istio-data\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.481462 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.481430 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx4br\" (UniqueName: \"kubernetes.io/projected/5d3df20a-43d3-437b-a707-616076c7455e-kube-api-access-lx4br\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.481462 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.481470 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d3df20a-43d3-437b-a707-616076c7455e-istio-token\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.481731 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.481488 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-istio-data\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.481731 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.481511 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-workload-certs\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.481731 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.481551 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d3df20a-43d3-437b-a707-616076c7455e-istiod-ca-cert\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.481731 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.481584 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-istio-envoy\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.481731 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.481631 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-credential-socket\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.481731 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.481670 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d3df20a-43d3-437b-a707-616076c7455e-istio-podinfo\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.481731 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.481715 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-workload-socket\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.482109 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.481952 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-istio-data\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.482109 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.482022 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-workload-certs\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.482192 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.482120 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-credential-socket\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.482234 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.482185 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-workload-socket\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.482280 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.482257 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d3df20a-43d3-437b-a707-616076c7455e-istiod-ca-cert\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.484176 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.484149 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-istio-envoy\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.484283 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.484266 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d3df20a-43d3-437b-a707-616076c7455e-istio-podinfo\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.489859 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.489838 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx4br\" (UniqueName: \"kubernetes.io/projected/5d3df20a-43d3-437b-a707-616076c7455e-kube-api-access-lx4br\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.489958 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.489864 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d3df20a-43d3-437b-a707-616076c7455e-istio-token\") pod \"e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.571986 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.571918 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:10.711292 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.711268 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95"] Jul 18 02:23:10.713429 ip-10-0-137-108 kubenswrapper[2548]: W0718 02:23:10.713404 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d3df20a_43d3_437b_a707_616076c7455e.slice/crio-d5797728e2c16fed559706638aa8417d9f81a85c986efe218840b70a67a21763 WatchSource:0}: Error finding container d5797728e2c16fed559706638aa8417d9f81a85c986efe218840b70a67a21763: Status 404 returned error can't find the container with id d5797728e2c16fed559706638aa8417d9f81a85c986efe218840b70a67a21763 Jul 18 02:23:10.715912 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.715882 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:23:10.715988 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.715946 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:23:10.715988 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:10.715976 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:23:11.295986 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:11.295951 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" event={"ID":"5d3df20a-43d3-437b-a707-616076c7455e","Type":"ContainerStarted","Data":"f0979887483dd6177c2413943b8e31f6c773dd486120cb31983ddd797c234f0b"} Jul 18 02:23:11.296349 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:11.296000 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" event={"ID":"5d3df20a-43d3-437b-a707-616076c7455e","Type":"ContainerStarted","Data":"d5797728e2c16fed559706638aa8417d9f81a85c986efe218840b70a67a21763"} Jul 18 02:23:11.318925 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:11.318876 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" podStartSLOduration=1.318862846 podStartE2EDuration="1.318862846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:23:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:23:11.317560945 +0000 UTC m=+2100.170304289" watchObservedRunningTime="2026-07-18 02:23:11.318862846 +0000 UTC m=+2100.171606189" Jul 18 02:23:11.572246 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:11.572177 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:11.573687 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:11.573659 2548 patch_prober.go:28] interesting pod/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 18 02:23:11.573857 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:11.573723 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" podUID="5d3df20a-43d3-437b-a707-616076c7455e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 18 02:23:11.681465 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:11.681438 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 02:23:11.712870 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:11.712845 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 02:23:12.572630 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:12.572595 2548 patch_prober.go:28] interesting pod/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 18 02:23:12.573024 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:12.572651 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" podUID="5d3df20a-43d3-437b-a707-616076c7455e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 18 02:23:13.576124 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:13.576095 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:14.311548 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:14.311517 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:14.312576 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:14.312556 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:29.313107 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:29.313074 2548 patch_prober.go:28] interesting pod/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 18 02:23:29.313495 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:29.313124 2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" podUID="5d3df20a-43d3-437b-a707-616076c7455e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 18 02:23:29.398104 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:29.398075 2548 generic.go:356] "Generic (PLEG): container finished" podID="5d3df20a-43d3-437b-a707-616076c7455e" containerID="f0979887483dd6177c2413943b8e31f6c773dd486120cb31983ddd797c234f0b" exitCode=0 Jul 18 02:23:29.398248 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:29.398138 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" event={"ID":"5d3df20a-43d3-437b-a707-616076c7455e","Type":"ContainerDied","Data":"f0979887483dd6177c2413943b8e31f6c773dd486120cb31983ddd797c234f0b"} Jul 18 02:23:29.398521 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:29.398507 2548 scope.go:117] "RemoveContainer" containerID="f0979887483dd6177c2413943b8e31f6c773dd486120cb31983ddd797c234f0b" Jul 18 02:23:29.399598 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:29.399568 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:23:29.399664 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:29.399650 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:23:29.399698 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:29.399686 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:23:30.405125 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:30.405091 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" event={"ID":"5d3df20a-43d3-437b-a707-616076c7455e","Type":"ContainerStarted","Data":"d5272fae5aec6b57791ca657e636439972dc949b417b72d94b51df5b9f9b328a"} Jul 18 02:23:30.572955 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:30.572914 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:30.573165 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:30.573128 2548 patch_prober.go:28] interesting pod/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 18 02:23:30.573239 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:30.573178 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" podUID="5d3df20a-43d3-437b-a707-616076c7455e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 18 02:23:31.573390 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:31.573351 2548 patch_prober.go:28] interesting pod/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 18 02:23:31.573810 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:31.573418 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" podUID="5d3df20a-43d3-437b-a707-616076c7455e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 18 02:23:32.572719 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:32.572683 2548 patch_prober.go:28] interesting pod/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 18 02:23:32.573051 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:32.573017 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" podUID="5d3df20a-43d3-437b-a707-616076c7455e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 18 02:23:33.573075 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:33.573034 2548 patch_prober.go:28] interesting pod/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 18 02:23:33.573440 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:33.573098 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" podUID="5d3df20a-43d3-437b-a707-616076c7455e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 18 02:23:34.576149 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:34.576118 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:34.576537 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:34.576468 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:23:34.577196 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:23:34.577178 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:25:02.043933 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:02.043895 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95"] Jul 18 02:25:02.044512 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:02.044113 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" podUID="5d3df20a-43d3-437b-a707-616076c7455e" containerName="istio-proxy" containerID="cri-o://d5272fae5aec6b57791ca657e636439972dc949b417b72d94b51df5b9f9b328a" gracePeriod=30 Jul 18 02:25:04.579092 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:04.579059 2548 patch_prober.go:28] interesting pod/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 18 02:25:04.579468 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:04.579117 2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" podUID="5d3df20a-43d3-437b-a707-616076c7455e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 18 02:25:07.293838 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.293811 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:25:07.299563 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.299543 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d3df20a-43d3-437b-a707-616076c7455e-istio-token\") pod \"5d3df20a-43d3-437b-a707-616076c7455e\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " Jul 18 02:25:07.299659 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.299584 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-credential-socket\") pod \"5d3df20a-43d3-437b-a707-616076c7455e\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " Jul 18 02:25:07.299659 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.299607 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-workload-socket\") pod \"5d3df20a-43d3-437b-a707-616076c7455e\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " Jul 18 02:25:07.299659 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.299624 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx4br\" (UniqueName: \"kubernetes.io/projected/5d3df20a-43d3-437b-a707-616076c7455e-kube-api-access-lx4br\") pod \"5d3df20a-43d3-437b-a707-616076c7455e\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " Jul 18 02:25:07.299659 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.299648 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d3df20a-43d3-437b-a707-616076c7455e-istiod-ca-cert\") pod \"5d3df20a-43d3-437b-a707-616076c7455e\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " Jul 18 02:25:07.299899 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.299665 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-istio-envoy\") pod \"5d3df20a-43d3-437b-a707-616076c7455e\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " Jul 18 02:25:07.299899 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.299703 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-workload-certs\") pod \"5d3df20a-43d3-437b-a707-616076c7455e\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " Jul 18 02:25:07.299899 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.299729 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d3df20a-43d3-437b-a707-616076c7455e-istio-podinfo\") pod \"5d3df20a-43d3-437b-a707-616076c7455e\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " Jul 18 02:25:07.299899 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.299754 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-istio-data\") pod \"5d3df20a-43d3-437b-a707-616076c7455e\" (UID: \"5d3df20a-43d3-437b-a707-616076c7455e\") " Jul 18 02:25:07.299899 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.299823 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5d3df20a-43d3-437b-a707-616076c7455e" (UID: "5d3df20a-43d3-437b-a707-616076c7455e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:07.299899 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.299890 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5d3df20a-43d3-437b-a707-616076c7455e" (UID: "5d3df20a-43d3-437b-a707-616076c7455e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:07.300219 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.300042 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5d3df20a-43d3-437b-a707-616076c7455e" (UID: "5d3df20a-43d3-437b-a707-616076c7455e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:07.300219 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.300071 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d3df20a-43d3-437b-a707-616076c7455e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5d3df20a-43d3-437b-a707-616076c7455e" (UID: "5d3df20a-43d3-437b-a707-616076c7455e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:25:07.300219 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.300119 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-workload-certs\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:25:07.300219 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.300138 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-credential-socket\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:25:07.300219 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.300153 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-workload-socket\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:25:07.300530 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.300490 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-istio-data" (OuterVolumeSpecName: "istio-data") pod "5d3df20a-43d3-437b-a707-616076c7455e" (UID: "5d3df20a-43d3-437b-a707-616076c7455e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:07.301746 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.301720 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d3df20a-43d3-437b-a707-616076c7455e-istio-token" (OuterVolumeSpecName: "istio-token") pod "5d3df20a-43d3-437b-a707-616076c7455e" (UID: "5d3df20a-43d3-437b-a707-616076c7455e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:25:07.301902 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.301877 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5d3df20a-43d3-437b-a707-616076c7455e" (UID: "5d3df20a-43d3-437b-a707-616076c7455e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:25:07.302153 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.302134 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5d3df20a-43d3-437b-a707-616076c7455e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5d3df20a-43d3-437b-a707-616076c7455e" (UID: "5d3df20a-43d3-437b-a707-616076c7455e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:25:07.302210 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.302188 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d3df20a-43d3-437b-a707-616076c7455e-kube-api-access-lx4br" (OuterVolumeSpecName: "kube-api-access-lx4br") pod "5d3df20a-43d3-437b-a707-616076c7455e" (UID: "5d3df20a-43d3-437b-a707-616076c7455e"). InnerVolumeSpecName "kube-api-access-lx4br". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:25:07.400849 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.400825 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx4br\" (UniqueName: \"kubernetes.io/projected/5d3df20a-43d3-437b-a707-616076c7455e-kube-api-access-lx4br\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:25:07.400849 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.400845 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5d3df20a-43d3-437b-a707-616076c7455e-istiod-ca-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:25:07.400849 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.400854 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-istio-envoy\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:25:07.401037 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.400862 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5d3df20a-43d3-437b-a707-616076c7455e-istio-podinfo\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:25:07.401037 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.400870 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5d3df20a-43d3-437b-a707-616076c7455e-istio-data\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:25:07.401037 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.400879 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5d3df20a-43d3-437b-a707-616076c7455e-istio-token\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:25:07.926312 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.926277 2548 generic.go:356] "Generic (PLEG): container finished" podID="5d3df20a-43d3-437b-a707-616076c7455e" containerID="d5272fae5aec6b57791ca657e636439972dc949b417b72d94b51df5b9f9b328a" exitCode=0 Jul 18 02:25:07.926483 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.926318 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" event={"ID":"5d3df20a-43d3-437b-a707-616076c7455e","Type":"ContainerDied","Data":"d5272fae5aec6b57791ca657e636439972dc949b417b72d94b51df5b9f9b328a"} Jul 18 02:25:07.926483 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.926337 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" Jul 18 02:25:07.926483 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.926374 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95" event={"ID":"5d3df20a-43d3-437b-a707-616076c7455e","Type":"ContainerDied","Data":"d5797728e2c16fed559706638aa8417d9f81a85c986efe218840b70a67a21763"} Jul 18 02:25:07.926483 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.926403 2548 scope.go:117] "RemoveContainer" containerID="d5272fae5aec6b57791ca657e636439972dc949b417b72d94b51df5b9f9b328a" Jul 18 02:25:07.935526 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.935506 2548 scope.go:117] "RemoveContainer" containerID="f0979887483dd6177c2413943b8e31f6c773dd486120cb31983ddd797c234f0b" Jul 18 02:25:07.943346 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.943328 2548 scope.go:117] "RemoveContainer" containerID="d5272fae5aec6b57791ca657e636439972dc949b417b72d94b51df5b9f9b328a" Jul 18 02:25:07.943578 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:25:07.943554 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5272fae5aec6b57791ca657e636439972dc949b417b72d94b51df5b9f9b328a\": container with ID starting with d5272fae5aec6b57791ca657e636439972dc949b417b72d94b51df5b9f9b328a not found: ID does not exist" containerID="d5272fae5aec6b57791ca657e636439972dc949b417b72d94b51df5b9f9b328a" Jul 18 02:25:07.943638 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.943586 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5272fae5aec6b57791ca657e636439972dc949b417b72d94b51df5b9f9b328a"} err="failed to get container status \"d5272fae5aec6b57791ca657e636439972dc949b417b72d94b51df5b9f9b328a\": rpc error: code = NotFound desc = could not find container \"d5272fae5aec6b57791ca657e636439972dc949b417b72d94b51df5b9f9b328a\": container with ID starting with d5272fae5aec6b57791ca657e636439972dc949b417b72d94b51df5b9f9b328a not found: ID does not exist" Jul 18 02:25:07.943638 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.943605 2548 scope.go:117] "RemoveContainer" containerID="f0979887483dd6177c2413943b8e31f6c773dd486120cb31983ddd797c234f0b" Jul 18 02:25:07.943897 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:25:07.943879 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0979887483dd6177c2413943b8e31f6c773dd486120cb31983ddd797c234f0b\": container with ID starting with f0979887483dd6177c2413943b8e31f6c773dd486120cb31983ddd797c234f0b not found: ID does not exist" containerID="f0979887483dd6177c2413943b8e31f6c773dd486120cb31983ddd797c234f0b" Jul 18 02:25:07.943957 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.943902 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0979887483dd6177c2413943b8e31f6c773dd486120cb31983ddd797c234f0b"} err="failed to get container status \"f0979887483dd6177c2413943b8e31f6c773dd486120cb31983ddd797c234f0b\": rpc error: code = NotFound desc = could not find container \"f0979887483dd6177c2413943b8e31f6c773dd486120cb31983ddd797c234f0b\": container with ID starting with f0979887483dd6177c2413943b8e31f6c773dd486120cb31983ddd797c234f0b not found: ID does not exist" Jul 18 02:25:07.944517 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.944500 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95"] Jul 18 02:25:07.948954 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:07.948935 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-8b6a54-openshift-default-65d9d5dd5c-fcp95"] Jul 18 02:25:09.549159 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:09.549129 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d3df20a-43d3-437b-a707-616076c7455e" path="/var/lib/kubelet/pods/5d3df20a-43d3-437b-a707-616076c7455e/volumes" Jul 18 02:25:24.703746 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.703705 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8"] Jul 18 02:25:24.704364 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.704341 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d3df20a-43d3-437b-a707-616076c7455e" containerName="istio-proxy" Jul 18 02:25:24.704444 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.704367 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3df20a-43d3-437b-a707-616076c7455e" containerName="istio-proxy" Jul 18 02:25:24.704444 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.704381 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d3df20a-43d3-437b-a707-616076c7455e" containerName="istio-proxy" Jul 18 02:25:24.704444 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.704390 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d3df20a-43d3-437b-a707-616076c7455e" containerName="istio-proxy" Jul 18 02:25:24.704597 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.704488 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d3df20a-43d3-437b-a707-616076c7455e" containerName="istio-proxy" Jul 18 02:25:24.704597 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.704501 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d3df20a-43d3-437b-a707-616076c7455e" containerName="istio-proxy" Jul 18 02:25:24.704597 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.704515 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c8d9594-f547-4033-915d-6fd8a61ee68e" containerName="istio-proxy" Jul 18 02:25:24.707907 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.707888 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.710519 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.710501 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-58b546-openshift-default-dockercfg-mzqn5\"" Jul 18 02:25:24.720849 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.720826 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8"] Jul 18 02:25:24.737693 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.737660 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-istio-data\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.737920 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.737709 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8be80b1d-7d29-4732-95fa-4c786073f916-istio-podinfo\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.737920 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.737808 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jrwd\" (UniqueName: \"kubernetes.io/projected/8be80b1d-7d29-4732-95fa-4c786073f916-kube-api-access-5jrwd\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.738069 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.737969 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-credential-socket\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.738069 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.738006 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8be80b1d-7d29-4732-95fa-4c786073f916-istiod-ca-cert\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.738069 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.738064 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-workload-certs\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.738223 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.738100 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-workload-socket\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.738223 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.738124 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8be80b1d-7d29-4732-95fa-4c786073f916-istio-token\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.738223 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.738160 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-istio-envoy\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.839113 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.839080 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-credential-socket\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.839113 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.839113 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8be80b1d-7d29-4732-95fa-4c786073f916-istiod-ca-cert\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.839362 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.839159 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-workload-certs\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.839362 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.839187 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-workload-socket\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.839362 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.839211 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8be80b1d-7d29-4732-95fa-4c786073f916-istio-token\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.839362 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.839243 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-istio-envoy\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.839362 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.839308 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-istio-data\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.839362 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.839340 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8be80b1d-7d29-4732-95fa-4c786073f916-istio-podinfo\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.839686 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.839382 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jrwd\" (UniqueName: \"kubernetes.io/projected/8be80b1d-7d29-4732-95fa-4c786073f916-kube-api-access-5jrwd\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.839686 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.839529 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-credential-socket\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.839686 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.839631 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-workload-socket\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.839686 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.839660 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-istio-data\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.839930 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.839814 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-workload-certs\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.839991 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.839947 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8be80b1d-7d29-4732-95fa-4c786073f916-istiod-ca-cert\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.841497 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.841476 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-istio-envoy\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.841792 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.841758 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8be80b1d-7d29-4732-95fa-4c786073f916-istio-podinfo\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.847847 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.847821 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jrwd\" (UniqueName: \"kubernetes.io/projected/8be80b1d-7d29-4732-95fa-4c786073f916-kube-api-access-5jrwd\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:24.848439 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:24.848422 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8be80b1d-7d29-4732-95fa-4c786073f916-istio-token\") pod \"e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:25.017677 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:25.017607 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:25.159079 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:25.159056 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8"] Jul 18 02:25:25.161062 ip-10-0-137-108 kubenswrapper[2548]: W0718 02:25:25.161034 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8be80b1d_7d29_4732_95fa_4c786073f916.slice/crio-5d82581793d34f6f0c55f583c86198ef7fc5e3ea918d067f99e3e6f3e4bfa2fb WatchSource:0}: Error finding container 5d82581793d34f6f0c55f583c86198ef7fc5e3ea918d067f99e3e6f3e4bfa2fb: Status 404 returned error can't find the container with id 5d82581793d34f6f0c55f583c86198ef7fc5e3ea918d067f99e3e6f3e4bfa2fb Jul 18 02:25:25.163226 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:25.163198 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:25:25.163296 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:25.163257 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:25:25.163296 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:25.163284 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:25:26.021518 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:26.021484 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" event={"ID":"8be80b1d-7d29-4732-95fa-4c786073f916","Type":"ContainerStarted","Data":"11b7d86fc61c63ff4d8013b76237d86463c1c4eed9c62c548f939c530132cabd"} Jul 18 02:25:26.021897 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:26.021534 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" event={"ID":"8be80b1d-7d29-4732-95fa-4c786073f916","Type":"ContainerStarted","Data":"5d82581793d34f6f0c55f583c86198ef7fc5e3ea918d067f99e3e6f3e4bfa2fb"} Jul 18 02:25:26.041373 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:26.041328 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" podStartSLOduration=2.041315379 podStartE2EDuration="2.041315379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:25:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:25:26.039552015 +0000 UTC m=+2234.892295360" watchObservedRunningTime="2026-07-18 02:25:26.041315379 +0000 UTC m=+2234.894058723" Jul 18 02:25:27.018424 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:27.018396 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:27.023507 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:27.023483 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:27.026899 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:27.026878 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:27.027548 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:27.027532 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:43.067932 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:43.067892 2548 patch_prober.go:28] interesting pod/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 18 02:25:43.068323 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:43.067964 2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" podUID="8be80b1d-7d29-4732-95fa-4c786073f916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 18 02:25:44.118079 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:44.118041 2548 generic.go:356] "Generic (PLEG): container finished" podID="8be80b1d-7d29-4732-95fa-4c786073f916" containerID="11b7d86fc61c63ff4d8013b76237d86463c1c4eed9c62c548f939c530132cabd" exitCode=0 Jul 18 02:25:44.118515 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:44.118112 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" event={"ID":"8be80b1d-7d29-4732-95fa-4c786073f916","Type":"ContainerDied","Data":"11b7d86fc61c63ff4d8013b76237d86463c1c4eed9c62c548f939c530132cabd"} Jul 18 02:25:44.118515 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:44.118475 2548 scope.go:117] "RemoveContainer" containerID="11b7d86fc61c63ff4d8013b76237d86463c1c4eed9c62c548f939c530132cabd" Jul 18 02:25:44.119446 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:44.119419 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:25:44.119499 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:44.119487 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:25:44.119535 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:44.119518 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:25:45.125735 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:45.125700 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" event={"ID":"8be80b1d-7d29-4732-95fa-4c786073f916","Type":"ContainerStarted","Data":"c8536de5328adc39d3c3c95cee8d508c48ad4fae6a5095bf7c3ae03255122e27"} Jul 18 02:25:46.018517 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:46.018484 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:46.018706 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:46.018682 2548 patch_prober.go:28] interesting pod/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 18 02:25:46.018759 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:46.018729 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" podUID="8be80b1d-7d29-4732-95fa-4c786073f916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 18 02:25:47.018857 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:47.018824 2548 patch_prober.go:28] interesting pod/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 18 02:25:47.019281 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:47.018889 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" podUID="8be80b1d-7d29-4732-95fa-4c786073f916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 18 02:25:48.018886 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:48.018848 2548 patch_prober.go:28] interesting pod/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 18 02:25:48.019264 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:48.018915 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" podUID="8be80b1d-7d29-4732-95fa-4c786073f916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 18 02:25:49.018284 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:49.018250 2548 patch_prober.go:28] interesting pod/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 18 02:25:49.018461 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:49.018305 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" podUID="8be80b1d-7d29-4732-95fa-4c786073f916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 18 02:25:50.022361 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:50.022333 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:50.022856 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:50.022833 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:25:50.023549 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:25:50.023529 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:26:14.801153 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:14.801117 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8"] Jul 18 02:26:14.801714 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:14.801377 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" podUID="8be80b1d-7d29-4732-95fa-4c786073f916" containerName="istio-proxy" containerID="cri-o://c8536de5328adc39d3c3c95cee8d508c48ad4fae6a5095bf7c3ae03255122e27" gracePeriod=30 Jul 18 02:26:20.073856 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.073800 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:26:20.141256 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.141230 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-istio-data\") pod \"8be80b1d-7d29-4732-95fa-4c786073f916\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " Jul 18 02:26:20.141387 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.141270 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8be80b1d-7d29-4732-95fa-4c786073f916-istio-token\") pod \"8be80b1d-7d29-4732-95fa-4c786073f916\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " Jul 18 02:26:20.141387 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.141316 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-workload-socket\") pod \"8be80b1d-7d29-4732-95fa-4c786073f916\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " Jul 18 02:26:20.141387 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.141365 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8be80b1d-7d29-4732-95fa-4c786073f916-istiod-ca-cert\") pod \"8be80b1d-7d29-4732-95fa-4c786073f916\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " Jul 18 02:26:20.141577 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.141449 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8be80b1d-7d29-4732-95fa-4c786073f916-istio-podinfo\") pod \"8be80b1d-7d29-4732-95fa-4c786073f916\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " Jul 18 02:26:20.141577 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.141480 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-istio-envoy\") pod \"8be80b1d-7d29-4732-95fa-4c786073f916\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " Jul 18 02:26:20.141577 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.141531 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-credential-socket\") pod \"8be80b1d-7d29-4732-95fa-4c786073f916\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " Jul 18 02:26:20.141577 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.141570 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-workload-certs\") pod \"8be80b1d-7d29-4732-95fa-4c786073f916\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " Jul 18 02:26:20.141902 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.141608 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8be80b1d-7d29-4732-95fa-4c786073f916" (UID: "8be80b1d-7d29-4732-95fa-4c786073f916"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:26:20.141902 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.141629 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jrwd\" (UniqueName: \"kubernetes.io/projected/8be80b1d-7d29-4732-95fa-4c786073f916-kube-api-access-5jrwd\") pod \"8be80b1d-7d29-4732-95fa-4c786073f916\" (UID: \"8be80b1d-7d29-4732-95fa-4c786073f916\") " Jul 18 02:26:20.141902 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.141706 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8be80b1d-7d29-4732-95fa-4c786073f916-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8be80b1d-7d29-4732-95fa-4c786073f916" (UID: "8be80b1d-7d29-4732-95fa-4c786073f916"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:26:20.142070 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.141994 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8be80b1d-7d29-4732-95fa-4c786073f916" (UID: "8be80b1d-7d29-4732-95fa-4c786073f916"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:26:20.142070 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.142001 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8be80b1d-7d29-4732-95fa-4c786073f916-istiod-ca-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:26:20.142070 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.142023 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-workload-socket\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:26:20.142237 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.142208 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-istio-data" (OuterVolumeSpecName: "istio-data") pod "8be80b1d-7d29-4732-95fa-4c786073f916" (UID: "8be80b1d-7d29-4732-95fa-4c786073f916"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:26:20.142297 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.142279 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8be80b1d-7d29-4732-95fa-4c786073f916" (UID: "8be80b1d-7d29-4732-95fa-4c786073f916"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:26:20.144054 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.144022 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8be80b1d-7d29-4732-95fa-4c786073f916" (UID: "8be80b1d-7d29-4732-95fa-4c786073f916"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:26:20.144054 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.144038 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8be80b1d-7d29-4732-95fa-4c786073f916-istio-token" (OuterVolumeSpecName: "istio-token") pod "8be80b1d-7d29-4732-95fa-4c786073f916" (UID: "8be80b1d-7d29-4732-95fa-4c786073f916"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:26:20.144220 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.144092 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8be80b1d-7d29-4732-95fa-4c786073f916-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8be80b1d-7d29-4732-95fa-4c786073f916" (UID: "8be80b1d-7d29-4732-95fa-4c786073f916"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:26:20.144220 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.144164 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8be80b1d-7d29-4732-95fa-4c786073f916-kube-api-access-5jrwd" (OuterVolumeSpecName: "kube-api-access-5jrwd") pod "8be80b1d-7d29-4732-95fa-4c786073f916" (UID: "8be80b1d-7d29-4732-95fa-4c786073f916"). InnerVolumeSpecName "kube-api-access-5jrwd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:26:20.243331 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.243300 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-istio-data\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:26:20.243331 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.243323 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8be80b1d-7d29-4732-95fa-4c786073f916-istio-token\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:26:20.243331 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.243331 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8be80b1d-7d29-4732-95fa-4c786073f916-istio-podinfo\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:26:20.243575 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.243342 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-istio-envoy\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:26:20.243575 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.243351 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-credential-socket\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:26:20.243575 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.243359 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8be80b1d-7d29-4732-95fa-4c786073f916-workload-certs\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:26:20.243575 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.243368 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jrwd\" (UniqueName: \"kubernetes.io/projected/8be80b1d-7d29-4732-95fa-4c786073f916-kube-api-access-5jrwd\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:26:20.314822 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.314770 2548 generic.go:356] "Generic (PLEG): container finished" podID="8be80b1d-7d29-4732-95fa-4c786073f916" containerID="c8536de5328adc39d3c3c95cee8d508c48ad4fae6a5095bf7c3ae03255122e27" exitCode=0 Jul 18 02:26:20.314943 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.314839 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" Jul 18 02:26:20.314943 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.314885 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" event={"ID":"8be80b1d-7d29-4732-95fa-4c786073f916","Type":"ContainerDied","Data":"c8536de5328adc39d3c3c95cee8d508c48ad4fae6a5095bf7c3ae03255122e27"} Jul 18 02:26:20.315018 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.314944 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" event={"ID":"8be80b1d-7d29-4732-95fa-4c786073f916","Type":"ContainerDied","Data":"5d82581793d34f6f0c55f583c86198ef7fc5e3ea918d067f99e3e6f3e4bfa2fb"} Jul 18 02:26:20.315018 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.314971 2548 scope.go:117] "RemoveContainer" containerID="c8536de5328adc39d3c3c95cee8d508c48ad4fae6a5095bf7c3ae03255122e27" Jul 18 02:26:20.324275 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.324257 2548 scope.go:117] "RemoveContainer" containerID="11b7d86fc61c63ff4d8013b76237d86463c1c4eed9c62c548f939c530132cabd" Jul 18 02:26:20.332060 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.332044 2548 scope.go:117] "RemoveContainer" containerID="c8536de5328adc39d3c3c95cee8d508c48ad4fae6a5095bf7c3ae03255122e27" Jul 18 02:26:20.332299 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:26:20.332279 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8536de5328adc39d3c3c95cee8d508c48ad4fae6a5095bf7c3ae03255122e27\": container with ID starting with c8536de5328adc39d3c3c95cee8d508c48ad4fae6a5095bf7c3ae03255122e27 not found: ID does not exist" containerID="c8536de5328adc39d3c3c95cee8d508c48ad4fae6a5095bf7c3ae03255122e27" Jul 18 02:26:20.332380 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.332308 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8536de5328adc39d3c3c95cee8d508c48ad4fae6a5095bf7c3ae03255122e27"} err="failed to get container status \"c8536de5328adc39d3c3c95cee8d508c48ad4fae6a5095bf7c3ae03255122e27\": rpc error: code = NotFound desc = could not find container \"c8536de5328adc39d3c3c95cee8d508c48ad4fae6a5095bf7c3ae03255122e27\": container with ID starting with c8536de5328adc39d3c3c95cee8d508c48ad4fae6a5095bf7c3ae03255122e27 not found: ID does not exist" Jul 18 02:26:20.332380 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.332329 2548 scope.go:117] "RemoveContainer" containerID="11b7d86fc61c63ff4d8013b76237d86463c1c4eed9c62c548f939c530132cabd" Jul 18 02:26:20.332565 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:26:20.332548 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11b7d86fc61c63ff4d8013b76237d86463c1c4eed9c62c548f939c530132cabd\": container with ID starting with 11b7d86fc61c63ff4d8013b76237d86463c1c4eed9c62c548f939c530132cabd not found: ID does not exist" containerID="11b7d86fc61c63ff4d8013b76237d86463c1c4eed9c62c548f939c530132cabd" Jul 18 02:26:20.332617 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.332570 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11b7d86fc61c63ff4d8013b76237d86463c1c4eed9c62c548f939c530132cabd"} err="failed to get container status \"11b7d86fc61c63ff4d8013b76237d86463c1c4eed9c62c548f939c530132cabd\": rpc error: code = NotFound desc = could not find container \"11b7d86fc61c63ff4d8013b76237d86463c1c4eed9c62c548f939c530132cabd\": container with ID starting with 11b7d86fc61c63ff4d8013b76237d86463c1c4eed9c62c548f939c530132cabd not found: ID does not exist" Jul 18 02:26:20.337986 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.337964 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8"] Jul 18 02:26:20.342693 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:20.342673 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8"] Jul 18 02:26:21.023105 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:21.023072 2548 patch_prober.go:28] interesting pod/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 18 02:26:21.023293 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:21.023157 2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-58b546-openshift-default-7d676f64c-4p4j8" podUID="8be80b1d-7d29-4732-95fa-4c786073f916" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 18 02:26:21.549601 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:21.549562 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8be80b1d-7d29-4732-95fa-4c786073f916" path="/var/lib/kubelet/pods/8be80b1d-7d29-4732-95fa-4c786073f916/volumes" Jul 18 02:26:42.455056 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.455022 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8"] Jul 18 02:26:42.455530 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.455463 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8be80b1d-7d29-4732-95fa-4c786073f916" containerName="istio-proxy" Jul 18 02:26:42.455530 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.455476 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="8be80b1d-7d29-4732-95fa-4c786073f916" containerName="istio-proxy" Jul 18 02:26:42.455646 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.455563 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="8be80b1d-7d29-4732-95fa-4c786073f916" containerName="istio-proxy" Jul 18 02:26:42.455646 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.455573 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="8be80b1d-7d29-4732-95fa-4c786073f916" containerName="istio-proxy" Jul 18 02:26:42.460307 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.460284 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.462723 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.462704 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-d05e31-openshift-default-dockercfg-4jp6z\"" Jul 18 02:26:42.470227 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.470203 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8"] Jul 18 02:26:42.638482 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.638451 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-token\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.638647 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.638523 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvtwn\" (UniqueName: \"kubernetes.io/projected/985a1fd6-c172-41de-9c68-c693f7b760a9-kube-api-access-qvtwn\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.638647 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.638562 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-credential-socket\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.638647 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.638593 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-workload-certs\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.638647 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.638613 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-envoy\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.638805 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.638656 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-podinfo\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.638805 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.638683 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-data\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.638805 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.638722 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-workload-socket\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.638805 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.638760 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/985a1fd6-c172-41de-9c68-c693f7b760a9-istiod-ca-cert\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.740130 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.740049 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvtwn\" (UniqueName: \"kubernetes.io/projected/985a1fd6-c172-41de-9c68-c693f7b760a9-kube-api-access-qvtwn\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.740130 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.740105 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-credential-socket\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.740350 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.740135 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-workload-certs\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.740350 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.740154 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-envoy\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.740350 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.740173 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-podinfo\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.740350 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.740200 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-data\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.740350 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.740277 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-workload-socket\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.740350 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.740336 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/985a1fd6-c172-41de-9c68-c693f7b760a9-istiod-ca-cert\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.740649 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.740415 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-token\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.740649 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.740567 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-workload-certs\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.740756 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.740743 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-data\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.740851 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.740773 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-credential-socket\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.740894 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.740851 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-workload-socket\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.741179 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.741156 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/985a1fd6-c172-41de-9c68-c693f7b760a9-istiod-ca-cert\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.742575 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.742519 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-envoy\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.743097 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.743074 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-podinfo\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.748853 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.748828 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-token\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.748953 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.748872 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvtwn\" (UniqueName: \"kubernetes.io/projected/985a1fd6-c172-41de-9c68-c693f7b760a9-kube-api-access-qvtwn\") pod \"e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.771507 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.771482 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:42.908981 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.908958 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8"] Jul 18 02:26:42.911057 ip-10-0-137-108 kubenswrapper[2548]: W0718 02:26:42.911029 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod985a1fd6_c172_41de_9c68_c693f7b760a9.slice/crio-741bf2ed410b08bd8dd75d56b2c895d8ab24936988c06d0925a60b09d0f96452 WatchSource:0}: Error finding container 741bf2ed410b08bd8dd75d56b2c895d8ab24936988c06d0925a60b09d0f96452: Status 404 returned error can't find the container with id 741bf2ed410b08bd8dd75d56b2c895d8ab24936988c06d0925a60b09d0f96452 Jul 18 02:26:42.913524 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.913488 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:26:42.913620 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.913552 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:26:42.913620 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:42.913580 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:26:43.429599 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:43.429557 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" event={"ID":"985a1fd6-c172-41de-9c68-c693f7b760a9","Type":"ContainerStarted","Data":"6c688369050410b44324371c63f58f4ce9b3dd025bc6f36fe52cde59d0334179"} Jul 18 02:26:43.429810 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:43.429620 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" event={"ID":"985a1fd6-c172-41de-9c68-c693f7b760a9","Type":"ContainerStarted","Data":"741bf2ed410b08bd8dd75d56b2c895d8ab24936988c06d0925a60b09d0f96452"} Jul 18 02:26:43.450592 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:43.450530 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" podStartSLOduration=1.4505140779999999 podStartE2EDuration="1.450514078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:26:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:26:43.447989062 +0000 UTC m=+2312.300732477" watchObservedRunningTime="2026-07-18 02:26:43.450514078 +0000 UTC m=+2312.303257498" Jul 18 02:26:43.772609 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:43.772524 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:43.777514 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:43.777486 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:44.435855 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:44.435825 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:44.437008 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:44.436989 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:26:59.601581 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:59.600225 2548 patch_prober.go:28] interesting pod/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": read tcp 10.132.0.2:33582->10.132.0.65:15021: read: connection reset by peer" start-of-body= Jul 18 02:26:59.601581 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:26:59.600284 2548 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" podUID="985a1fd6-c172-41de-9c68-c693f7b760a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": read tcp 10.132.0.2:33582->10.132.0.65:15021: read: connection reset by peer" Jul 18 02:27:00.530553 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:00.524360 2548 generic.go:356] "Generic (PLEG): container finished" podID="985a1fd6-c172-41de-9c68-c693f7b760a9" containerID="6c688369050410b44324371c63f58f4ce9b3dd025bc6f36fe52cde59d0334179" exitCode=0 Jul 18 02:27:00.530553 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:00.524531 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" event={"ID":"985a1fd6-c172-41de-9c68-c693f7b760a9","Type":"ContainerDied","Data":"6c688369050410b44324371c63f58f4ce9b3dd025bc6f36fe52cde59d0334179"} Jul 18 02:27:00.530553 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:00.525226 2548 scope.go:117] "RemoveContainer" containerID="6c688369050410b44324371c63f58f4ce9b3dd025bc6f36fe52cde59d0334179" Jul 18 02:27:00.531275 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:00.531245 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:27:00.531354 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:00.531336 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:27:00.531401 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:00.531387 2548 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 18 02:27:01.532964 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:01.532927 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" event={"ID":"985a1fd6-c172-41de-9c68-c693f7b760a9","Type":"ContainerStarted","Data":"ec9ec664c4f37b03a3d4ad9c4d1a4761c4761837303ceb8f01ff4caca7db62f2"} Jul 18 02:27:01.772453 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:01.772411 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:27:01.772724 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:01.772697 2548 patch_prober.go:28] interesting pod/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 18 02:27:01.772831 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:01.772759 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" podUID="985a1fd6-c172-41de-9c68-c693f7b760a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 18 02:27:02.772381 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:02.772342 2548 patch_prober.go:28] interesting pod/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 18 02:27:02.772808 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:02.772414 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" podUID="985a1fd6-c172-41de-9c68-c693f7b760a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 18 02:27:03.772304 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:03.772214 2548 patch_prober.go:28] interesting pod/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 18 02:27:03.772304 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:03.772274 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" podUID="985a1fd6-c172-41de-9c68-c693f7b760a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 18 02:27:04.772185 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:04.772142 2548 patch_prober.go:28] interesting pod/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 18 02:27:04.772369 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:04.772250 2548 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" podUID="985a1fd6-c172-41de-9c68-c693f7b760a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 18 02:27:05.775230 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:05.775193 2548 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:27:05.775712 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:05.775577 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:27:05.776138 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:27:05.776120 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:28:11.723307 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:28:11.723278 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 02:28:11.748664 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:28:11.748640 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 02:30:00.149638 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:30:00.149599 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739030-zsbcq"] Jul 18 02:30:00.150263 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:30:00.150242 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8be80b1d-7d29-4732-95fa-4c786073f916" containerName="istio-proxy" Jul 18 02:30:00.150343 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:30:00.150267 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="8be80b1d-7d29-4732-95fa-4c786073f916" containerName="istio-proxy" Jul 18 02:30:00.153532 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:30:00.153509 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739030-zsbcq" Jul 18 02:30:00.156012 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:30:00.155993 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dkjtj\"" Jul 18 02:30:00.167206 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:30:00.166687 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739030-zsbcq"] Jul 18 02:30:00.167206 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:30:00.167167 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739030-zsbcq" Jul 18 02:30:00.510073 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:30:00.509989 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29739030-zsbcq"] Jul 18 02:30:00.514181 ip-10-0-137-108 kubenswrapper[2548]: W0718 02:30:00.514157 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32fa3b5d_5f89_4eb6_839f_7b355b461fd6.slice/crio-b67448c21074ccd90ee07d7e6fb0835b733b20f25b1f8eea4ea160559986cff9 WatchSource:0}: Error finding container b67448c21074ccd90ee07d7e6fb0835b733b20f25b1f8eea4ea160559986cff9: Status 404 returned error can't find the container with id b67448c21074ccd90ee07d7e6fb0835b733b20f25b1f8eea4ea160559986cff9 Jul 18 02:30:01.469527 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:30:01.469499 2548 generic.go:356] "Generic (PLEG): container finished" podID="32fa3b5d-5f89-4eb6-839f-7b355b461fd6" containerID="c9594bbc3092655f633fa83abc19835967e1485727f4a7b357164c914e181fb9" exitCode=0 Jul 18 02:30:01.469853 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:30:01.469579 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739030-zsbcq" event={"ID":"32fa3b5d-5f89-4eb6-839f-7b355b461fd6","Type":"ContainerDied","Data":"c9594bbc3092655f633fa83abc19835967e1485727f4a7b357164c914e181fb9"} Jul 18 02:30:01.469853 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:30:01.469629 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739030-zsbcq" event={"ID":"32fa3b5d-5f89-4eb6-839f-7b355b461fd6","Type":"ContainerStarted","Data":"b67448c21074ccd90ee07d7e6fb0835b733b20f25b1f8eea4ea160559986cff9"} Jul 18 02:30:02.626856 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:30:02.626817 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739030-zsbcq" Jul 18 02:30:03.482706 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:30:03.482627 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739030-zsbcq" Jul 18 02:30:03.482706 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:30:03.482671 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29739030-zsbcq" event={"ID":"32fa3b5d-5f89-4eb6-839f-7b355b461fd6","Type":"ContainerDied","Data":"b67448c21074ccd90ee07d7e6fb0835b733b20f25b1f8eea4ea160559986cff9"} Jul 18 02:30:03.482928 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:30:03.482717 2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b67448c21074ccd90ee07d7e6fb0835b733b20f25b1f8eea4ea160559986cff9" Jul 18 02:33:06.387190 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:06.387105 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7434-140083572529968"] Jul 18 02:33:06.387708 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:06.387685 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32fa3b5d-5f89-4eb6-839f-7b355b461fd6" containerName="cleanup" Jul 18 02:33:06.387814 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:06.387712 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="32fa3b5d-5f89-4eb6-839f-7b355b461fd6" containerName="cleanup" Jul 18 02:33:06.387886 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:06.387873 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="32fa3b5d-5f89-4eb6-839f-7b355b461fd6" containerName="cleanup" Jul 18 02:33:06.391041 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:06.391021 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572529968" Jul 18 02:33:06.393554 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:06.393531 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-tkmd9\"" Jul 18 02:33:06.395483 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:06.395461 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7434-140083572529968"] Jul 18 02:33:06.402700 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:06.402676 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmdq4\" (UniqueName: \"kubernetes.io/projected/bfc461c0-29c1-4aa9-9fc1-d1129d18eaee-kube-api-access-fmdq4\") pod \"test-curl-7434-140083572529968\" (UID: \"bfc461c0-29c1-4aa9-9fc1-d1129d18eaee\") " pod="opendatahub/test-curl-7434-140083572529968" Jul 18 02:33:06.503132 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:06.503107 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmdq4\" (UniqueName: \"kubernetes.io/projected/bfc461c0-29c1-4aa9-9fc1-d1129d18eaee-kube-api-access-fmdq4\") pod \"test-curl-7434-140083572529968\" (UID: \"bfc461c0-29c1-4aa9-9fc1-d1129d18eaee\") " pod="opendatahub/test-curl-7434-140083572529968" Jul 18 02:33:06.511548 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:06.511529 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmdq4\" (UniqueName: \"kubernetes.io/projected/bfc461c0-29c1-4aa9-9fc1-d1129d18eaee-kube-api-access-fmdq4\") pod \"test-curl-7434-140083572529968\" (UID: \"bfc461c0-29c1-4aa9-9fc1-d1129d18eaee\") " pod="opendatahub/test-curl-7434-140083572529968" Jul 18 02:33:06.701141 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:06.701110 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572529968" Jul 18 02:33:06.836701 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:06.836596 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7434-140083572529968"] Jul 18 02:33:06.839447 ip-10-0-137-108 kubenswrapper[2548]: W0718 02:33:06.839421 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbfc461c0_29c1_4aa9_9fc1_d1129d18eaee.slice/crio-d15715af8429b58054b877fbe129196daf28128d1605fc1d942f19f2960d993f WatchSource:0}: Error finding container d15715af8429b58054b877fbe129196daf28128d1605fc1d942f19f2960d993f: Status 404 returned error can't find the container with id d15715af8429b58054b877fbe129196daf28128d1605fc1d942f19f2960d993f Jul 18 02:33:06.840813 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:06.840760 2548 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 18 02:33:07.471939 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:07.471905 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7434-140083572529968" event={"ID":"bfc461c0-29c1-4aa9-9fc1-d1129d18eaee","Type":"ContainerStarted","Data":"d15715af8429b58054b877fbe129196daf28128d1605fc1d942f19f2960d993f"} Jul 18 02:33:08.482501 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:08.482461 2548 generic.go:356] "Generic (PLEG): container finished" podID="bfc461c0-29c1-4aa9-9fc1-d1129d18eaee" containerID="11bf94df27f77694ef465d73709d23c18ed6161e0914b66531fc3bfc1a6891b0" exitCode=0 Jul 18 02:33:08.482935 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:08.482549 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7434-140083572529968" event={"ID":"bfc461c0-29c1-4aa9-9fc1-d1129d18eaee","Type":"ContainerDied","Data":"11bf94df27f77694ef465d73709d23c18ed6161e0914b66531fc3bfc1a6891b0"} Jul 18 02:33:09.630626 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:09.630604 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572529968" Jul 18 02:33:09.665989 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:09.665955 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7434-140083572529968_bfc461c0-29c1-4aa9-9fc1-d1129d18eaee/test-curl-7434-140083572529968/0.log" Jul 18 02:33:09.688886 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:09.688856 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7434-140083572529968"] Jul 18 02:33:09.692998 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:09.692976 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7434-140083572529968"] Jul 18 02:33:09.735201 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:09.735181 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmdq4\" (UniqueName: \"kubernetes.io/projected/bfc461c0-29c1-4aa9-9fc1-d1129d18eaee-kube-api-access-fmdq4\") pod \"bfc461c0-29c1-4aa9-9fc1-d1129d18eaee\" (UID: \"bfc461c0-29c1-4aa9-9fc1-d1129d18eaee\") " Jul 18 02:33:09.737192 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:09.737168 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfc461c0-29c1-4aa9-9fc1-d1129d18eaee-kube-api-access-fmdq4" (OuterVolumeSpecName: "kube-api-access-fmdq4") pod "bfc461c0-29c1-4aa9-9fc1-d1129d18eaee" (UID: "bfc461c0-29c1-4aa9-9fc1-d1129d18eaee"). InnerVolumeSpecName "kube-api-access-fmdq4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:33:09.836616 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:09.836590 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmdq4\" (UniqueName: \"kubernetes.io/projected/bfc461c0-29c1-4aa9-9fc1-d1129d18eaee-kube-api-access-fmdq4\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:33:10.495195 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:10.495160 2548 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d15715af8429b58054b877fbe129196daf28128d1605fc1d942f19f2960d993f" Jul 18 02:33:10.495195 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:10.495180 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7434-140083572529968" Jul 18 02:33:10.505216 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:33:10.505189 2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7434-140083572529968\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="bfc461c0-29c1-4aa9-9fc1-d1129d18eaee" pod="opendatahub/test-curl-7434-140083572529968" Jul 18 02:33:11.548402 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:33:11.548367 2548 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7434-140083572529968\" is forbidden: User \"system:node:ip-10-0-137-108.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-108.ec2.internal' and this object" podUID="bfc461c0-29c1-4aa9-9fc1-d1129d18eaee" pod="opendatahub/test-curl-7434-140083572529968" Jul 18 02:33:11.549513 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:11.549489 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfc461c0-29c1-4aa9-9fc1-d1129d18eaee" path="/var/lib/kubelet/pods/bfc461c0-29c1-4aa9-9fc1-d1129d18eaee/volumes" Jul 18 02:33:11.755564 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:11.755540 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 02:33:11.788807 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:11.788765 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 02:33:43.445895 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:43.445764 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8"] Jul 18 02:33:43.446508 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:43.446064 2548 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" podUID="985a1fd6-c172-41de-9c68-c693f7b760a9" containerName="istio-proxy" containerID="cri-o://ec9ec664c4f37b03a3d4ad9c4d1a4761c4761837303ceb8f01ff4caca7db62f2" gracePeriod=30 Jul 18 02:33:48.685929 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.685904 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:33:48.694656 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.694632 2548 generic.go:356] "Generic (PLEG): container finished" podID="985a1fd6-c172-41de-9c68-c693f7b760a9" containerID="ec9ec664c4f37b03a3d4ad9c4d1a4761c4761837303ceb8f01ff4caca7db62f2" exitCode=0 Jul 18 02:33:48.694749 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.694674 2548 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" Jul 18 02:33:48.694749 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.694714 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" event={"ID":"985a1fd6-c172-41de-9c68-c693f7b760a9","Type":"ContainerDied","Data":"ec9ec664c4f37b03a3d4ad9c4d1a4761c4761837303ceb8f01ff4caca7db62f2"} Jul 18 02:33:48.694866 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.694807 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8" event={"ID":"985a1fd6-c172-41de-9c68-c693f7b760a9","Type":"ContainerDied","Data":"741bf2ed410b08bd8dd75d56b2c895d8ab24936988c06d0925a60b09d0f96452"} Jul 18 02:33:48.694866 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.694831 2548 scope.go:117] "RemoveContainer" containerID="ec9ec664c4f37b03a3d4ad9c4d1a4761c4761837303ceb8f01ff4caca7db62f2" Jul 18 02:33:48.703358 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.703337 2548 scope.go:117] "RemoveContainer" containerID="6c688369050410b44324371c63f58f4ce9b3dd025bc6f36fe52cde59d0334179" Jul 18 02:33:48.714273 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.714254 2548 scope.go:117] "RemoveContainer" containerID="ec9ec664c4f37b03a3d4ad9c4d1a4761c4761837303ceb8f01ff4caca7db62f2" Jul 18 02:33:48.714593 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:33:48.714567 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec9ec664c4f37b03a3d4ad9c4d1a4761c4761837303ceb8f01ff4caca7db62f2\": container with ID starting with ec9ec664c4f37b03a3d4ad9c4d1a4761c4761837303ceb8f01ff4caca7db62f2 not found: ID does not exist" containerID="ec9ec664c4f37b03a3d4ad9c4d1a4761c4761837303ceb8f01ff4caca7db62f2" Jul 18 02:33:48.714663 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.714604 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec9ec664c4f37b03a3d4ad9c4d1a4761c4761837303ceb8f01ff4caca7db62f2"} err="failed to get container status \"ec9ec664c4f37b03a3d4ad9c4d1a4761c4761837303ceb8f01ff4caca7db62f2\": rpc error: code = NotFound desc = could not find container \"ec9ec664c4f37b03a3d4ad9c4d1a4761c4761837303ceb8f01ff4caca7db62f2\": container with ID starting with ec9ec664c4f37b03a3d4ad9c4d1a4761c4761837303ceb8f01ff4caca7db62f2 not found: ID does not exist" Jul 18 02:33:48.714663 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.714628 2548 scope.go:117] "RemoveContainer" containerID="6c688369050410b44324371c63f58f4ce9b3dd025bc6f36fe52cde59d0334179" Jul 18 02:33:48.714942 ip-10-0-137-108 kubenswrapper[2548]: E0718 02:33:48.714923 2548 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c688369050410b44324371c63f58f4ce9b3dd025bc6f36fe52cde59d0334179\": container with ID starting with 6c688369050410b44324371c63f58f4ce9b3dd025bc6f36fe52cde59d0334179 not found: ID does not exist" containerID="6c688369050410b44324371c63f58f4ce9b3dd025bc6f36fe52cde59d0334179" Jul 18 02:33:48.715008 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.714947 2548 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c688369050410b44324371c63f58f4ce9b3dd025bc6f36fe52cde59d0334179"} err="failed to get container status \"6c688369050410b44324371c63f58f4ce9b3dd025bc6f36fe52cde59d0334179\": rpc error: code = NotFound desc = could not find container \"6c688369050410b44324371c63f58f4ce9b3dd025bc6f36fe52cde59d0334179\": container with ID starting with 6c688369050410b44324371c63f58f4ce9b3dd025bc6f36fe52cde59d0334179 not found: ID does not exist" Jul 18 02:33:48.879247 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.879223 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-workload-socket\") pod \"985a1fd6-c172-41de-9c68-c693f7b760a9\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " Jul 18 02:33:48.879388 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.879266 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-data\") pod \"985a1fd6-c172-41de-9c68-c693f7b760a9\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " Jul 18 02:33:48.879388 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.879299 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-token\") pod \"985a1fd6-c172-41de-9c68-c693f7b760a9\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " Jul 18 02:33:48.879388 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.879314 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/985a1fd6-c172-41de-9c68-c693f7b760a9-istiod-ca-cert\") pod \"985a1fd6-c172-41de-9c68-c693f7b760a9\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " Jul 18 02:33:48.879388 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.879343 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvtwn\" (UniqueName: \"kubernetes.io/projected/985a1fd6-c172-41de-9c68-c693f7b760a9-kube-api-access-qvtwn\") pod \"985a1fd6-c172-41de-9c68-c693f7b760a9\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " Jul 18 02:33:48.879388 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.879366 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-envoy\") pod \"985a1fd6-c172-41de-9c68-c693f7b760a9\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " Jul 18 02:33:48.879656 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.879455 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-workload-certs\") pod \"985a1fd6-c172-41de-9c68-c693f7b760a9\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " Jul 18 02:33:48.879656 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.879480 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-podinfo\") pod \"985a1fd6-c172-41de-9c68-c693f7b760a9\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " Jul 18 02:33:48.879656 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.879511 2548 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-credential-socket\") pod \"985a1fd6-c172-41de-9c68-c693f7b760a9\" (UID: \"985a1fd6-c172-41de-9c68-c693f7b760a9\") " Jul 18 02:33:48.879858 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.879506 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "985a1fd6-c172-41de-9c68-c693f7b760a9" (UID: "985a1fd6-c172-41de-9c68-c693f7b760a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:33:48.879858 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.879718 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "985a1fd6-c172-41de-9c68-c693f7b760a9" (UID: "985a1fd6-c172-41de-9c68-c693f7b760a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:33:48.879968 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.879865 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/985a1fd6-c172-41de-9c68-c693f7b760a9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "985a1fd6-c172-41de-9c68-c693f7b760a9" (UID: "985a1fd6-c172-41de-9c68-c693f7b760a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 18 02:33:48.880255 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.880200 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "985a1fd6-c172-41de-9c68-c693f7b760a9" (UID: "985a1fd6-c172-41de-9c68-c693f7b760a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:33:48.880374 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.880299 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-data" (OuterVolumeSpecName: "istio-data") pod "985a1fd6-c172-41de-9c68-c693f7b760a9" (UID: "985a1fd6-c172-41de-9c68-c693f7b760a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:33:48.881437 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.881411 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-token" (OuterVolumeSpecName: "istio-token") pod "985a1fd6-c172-41de-9c68-c693f7b760a9" (UID: "985a1fd6-c172-41de-9c68-c693f7b760a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:33:48.881687 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.881664 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/985a1fd6-c172-41de-9c68-c693f7b760a9-kube-api-access-qvtwn" (OuterVolumeSpecName: "kube-api-access-qvtwn") pod "985a1fd6-c172-41de-9c68-c693f7b760a9" (UID: "985a1fd6-c172-41de-9c68-c693f7b760a9"). InnerVolumeSpecName "kube-api-access-qvtwn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 18 02:33:48.881854 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.881831 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "985a1fd6-c172-41de-9c68-c693f7b760a9" (UID: "985a1fd6-c172-41de-9c68-c693f7b760a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 18 02:33:48.881911 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.881836 2548 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "985a1fd6-c172-41de-9c68-c693f7b760a9" (UID: "985a1fd6-c172-41de-9c68-c693f7b760a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 18 02:33:48.981181 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.981113 2548 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-data\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:33:48.981181 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.981136 2548 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-token\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:33:48.981181 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.981145 2548 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/985a1fd6-c172-41de-9c68-c693f7b760a9-istiod-ca-cert\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:33:48.981181 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.981154 2548 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvtwn\" (UniqueName: \"kubernetes.io/projected/985a1fd6-c172-41de-9c68-c693f7b760a9-kube-api-access-qvtwn\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:33:48.981181 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.981163 2548 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-envoy\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:33:48.981181 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.981172 2548 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-workload-certs\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:33:48.981181 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.981179 2548 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/985a1fd6-c172-41de-9c68-c693f7b760a9-istio-podinfo\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:33:48.981181 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.981187 2548 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-credential-socket\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:33:48.981521 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:48.981196 2548 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/985a1fd6-c172-41de-9c68-c693f7b760a9-workload-socket\") on node \"ip-10-0-137-108.ec2.internal\" DevicePath \"\"" Jul 18 02:33:49.018320 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:49.018292 2548 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8"] Jul 18 02:33:49.023972 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:49.023951 2548 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-d05e31-openshift-default-5768d48499-wbqm8"] Jul 18 02:33:49.548408 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:33:49.548378 2548 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="985a1fd6-c172-41de-9c68-c693f7b760a9" path="/var/lib/kubelet/pods/985a1fd6-c172-41de-9c68-c693f7b760a9/volumes" Jul 18 02:34:05.966761 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:05.966733 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f4847d7ff-k8pfq_1871e809-38a6-44e0-8e6c-a2317d250f6f/authorino/0.log" Jul 18 02:34:11.141437 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:11.141399 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-b8nbn_8a20675e-03ef-44a5-8663-ead09b5e6774/manager/0.log" Jul 18 02:34:11.255695 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:11.255612 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pvq8t_c0e8558a-a0ba-4d9c-a4bc-4cab55db2a77/manager/0.log" Jul 18 02:34:11.368573 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:11.368542 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-75667cc8f5-q7hjf_1fa4f31e-6ff9-4085-b65d-f8c04aa7d17a/manager/0.log" Jul 18 02:34:11.475941 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:11.475906 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-bk8lx_bca4080e-2f9e-453e-8d6b-6179b8665cfd/server/0.log" Jul 18 02:34:11.586447 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:11.586421 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-g57qp_953b02cb-cd60-4aa0-b742-e5035254d741/manager/0.log" Jul 18 02:34:11.693272 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:11.693244 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c588c4c7-6wk76_6a9da053-d6e0-4e23-bbf0-1765e1ded023/copy-manifests/0.log" Jul 18 02:34:11.704678 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:11.704655 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c588c4c7-6wk76_6a9da053-d6e0-4e23-bbf0-1765e1ded023/manager/0.log" Jul 18 02:34:12.268027 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:12.267993 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6576d4df98-sqdwl_edb6f98f-114b-4bcf-a091-a975c6bcdc39/maas-api/0.log" Jul 18 02:34:12.376448 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:12.376419 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29739000-hk26c_61da4321-a86f-4391-9632-d566b514fad2/cleanup/0.log" Jul 18 02:34:12.480877 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:12.480850 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29739015-77zdn_95b63bca-e55c-4e6c-8043-25a8ac2fa33d/cleanup/0.log" Jul 18 02:34:12.587031 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:12.586958 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29739030-zsbcq_32fa3b5d-5f89-4eb6-839f-7b355b461fd6/cleanup/0.log" Jul 18 02:34:12.696038 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:12.696009 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-nq7v6_6019d639-5f08-4d25-a871-29197b9334dd/postgres/0.log" Jul 18 02:34:13.923899 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:13.923873 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f4847d7ff-k8pfq_1871e809-38a6-44e0-8e6c-a2317d250f6f/authorino/0.log" Jul 18 02:34:14.037885 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:14.037861 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-x7bs7_e0ab8e91-24ee-4c70-bf9b-e6707293fb03/manager/0.log" Jul 18 02:34:14.142789 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:14.142745 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-zz8tb_7a028ff6-cd86-4e61-bca8-aee5c7d8cdb8/manager/0.log" Jul 18 02:34:14.249251 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:14.249179 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9nfft_806dc654-0543-4b3b-baec-68d01a5db966/kuadrant-console-plugin/0.log" Jul 18 02:34:14.360610 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:14.360573 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-k4dhq_3045c3c3-9924-4079-a61e-37accc8c1e6d/registry-server/0.log" Jul 18 02:34:14.470442 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:14.470414 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-9hfmw_7b2878f6-42fb-4b8b-bc1f-d2102423d6ec/manager/0.log" Jul 18 02:34:15.245828 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:15.245801 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c65d4b6f-g5q5h_56701e43-78f7-4acb-8f31-b3f75fc147af/kube-auth-proxy/0.log" Jul 18 02:34:15.467868 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:15.467838 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-62vpz_aeddc1c9-3b8b-4f5b-91a8-ac2655950649/istio-proxy/1.log" Jul 18 02:34:15.692759 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:15.692726 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-v8tzt_6da9d422-36df-404f-a158-d3de9a1b6a6a/payload-processing/0.log" Jul 18 02:34:15.799354 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:15.799325 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-84775f69dd-wq2kv_27d13595-ae03-4459-aa8c-546b1c4500b0/router/0.log" Jul 18 02:34:16.123335 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:16.123260 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-rxzd6_e3008b30-7e0b-455e-a326-e322ec934289/main/0.log" Jul 18 02:34:16.341490 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:16.341465 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-mqssc_e82e0f44-cb4e-4d1c-bc8a-e5f2d9d48973/main/0.log" Jul 18 02:34:16.675480 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:16.675450 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-thh2x_acfe2150-68c0-4f24-bb4d-73a658ea6068/main/0.log" Jul 18 02:34:24.378922 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:24.378886 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-525n9_79e51bb5-b3d5-4003-8c70-f789f06b54e8/global-pull-secret-syncer/0.log" Jul 18 02:34:24.562294 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:24.562265 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cdhlp_6d4e467b-f727-41ee-94b4-eb64e672a7dc/konnectivity-agent/0.log" Jul 18 02:34:24.611152 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:24.611127 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-108.ec2.internal_3d448a47aa0f5878864b37d16dc8c112/haproxy/0.log" Jul 18 02:34:30.195254 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:30.195227 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5f4847d7ff-k8pfq_1871e809-38a6-44e0-8e6c-a2317d250f6f/authorino/0.log" Jul 18 02:34:30.238615 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:30.238588 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-x7bs7_e0ab8e91-24ee-4c70-bf9b-e6707293fb03/manager/0.log" Jul 18 02:34:30.267720 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:30.267695 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-zz8tb_7a028ff6-cd86-4e61-bca8-aee5c7d8cdb8/manager/0.log" Jul 18 02:34:30.299301 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:30.299279 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-9nfft_806dc654-0543-4b3b-baec-68d01a5db966/kuadrant-console-plugin/0.log" Jul 18 02:34:30.341306 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:30.341284 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-k4dhq_3045c3c3-9924-4079-a61e-37accc8c1e6d/registry-server/0.log" Jul 18 02:34:30.457958 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:30.457877 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-9hfmw_7b2878f6-42fb-4b8b-bc1f-d2102423d6ec/manager/0.log" Jul 18 02:34:32.590554 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:32.590518 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-w98tz_b7250c41-3b61-4533-90d6-7382a4ea66c4/cluster-monitoring-operator/0.log" Jul 18 02:34:32.758772 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:32.758743 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ngj6b_9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc/node-exporter/0.log" Jul 18 02:34:32.782687 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:32.782661 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ngj6b_9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc/kube-rbac-proxy/0.log" Jul 18 02:34:32.806448 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:32.806429 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ngj6b_9a5e6c2b-2b9b-4ef1-ac2f-61eef2fac8bc/init-textfile/0.log" Jul 18 02:34:33.296494 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:33.296426 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f7bc69b77-cb2p8_9c8d03cd-5411-4f24-825c-0ec7735d4672/telemeter-client/0.log" Jul 18 02:34:33.315291 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:33.315268 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f7bc69b77-cb2p8_9c8d03cd-5411-4f24-825c-0ec7735d4672/reload/0.log" Jul 18 02:34:33.335162 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:33.335144 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f7bc69b77-cb2p8_9c8d03cd-5411-4f24-825c-0ec7735d4672/kube-rbac-proxy/0.log" Jul 18 02:34:34.329330 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.329296 2548 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k"] Jul 18 02:34:34.329687 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.329671 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="985a1fd6-c172-41de-9c68-c693f7b760a9" containerName="istio-proxy" Jul 18 02:34:34.329687 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.329683 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="985a1fd6-c172-41de-9c68-c693f7b760a9" containerName="istio-proxy" Jul 18 02:34:34.329759 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.329693 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfc461c0-29c1-4aa9-9fc1-d1129d18eaee" containerName="test-curl-7434-140083572529968" Jul 18 02:34:34.329759 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.329699 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfc461c0-29c1-4aa9-9fc1-d1129d18eaee" containerName="test-curl-7434-140083572529968" Jul 18 02:34:34.329759 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.329719 2548 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="985a1fd6-c172-41de-9c68-c693f7b760a9" containerName="istio-proxy" Jul 18 02:34:34.329759 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.329726 2548 state_mem.go:107] "Deleted CPUSet assignment" podUID="985a1fd6-c172-41de-9c68-c693f7b760a9" containerName="istio-proxy" Jul 18 02:34:34.329915 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.329799 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="985a1fd6-c172-41de-9c68-c693f7b760a9" containerName="istio-proxy" Jul 18 02:34:34.329915 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.329807 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="985a1fd6-c172-41de-9c68-c693f7b760a9" containerName="istio-proxy" Jul 18 02:34:34.329915 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.329812 2548 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfc461c0-29c1-4aa9-9fc1-d1129d18eaee" containerName="test-curl-7434-140083572529968" Jul 18 02:34:34.333041 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.333021 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:34.336158 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.336138 2548 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-28rtf\"/\"default-dockercfg-5hl2v\"" Jul 18 02:34:34.336270 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.336252 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-28rtf\"/\"openshift-service-ca.crt\"" Jul 18 02:34:34.336373 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.336350 2548 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-28rtf\"/\"kube-root-ca.crt\"" Jul 18 02:34:34.342724 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.342702 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k"] Jul 18 02:34:34.478772 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.478742 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpkqp\" (UniqueName: \"kubernetes.io/projected/4fb60fab-5cca-4668-b4c2-a469e4653f59-kube-api-access-vpkqp\") pod \"perf-node-gather-daemonset-s6x7k\" (UID: \"4fb60fab-5cca-4668-b4c2-a469e4653f59\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:34.478942 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.478827 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fb60fab-5cca-4668-b4c2-a469e4653f59-podres\") pod \"perf-node-gather-daemonset-s6x7k\" (UID: \"4fb60fab-5cca-4668-b4c2-a469e4653f59\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:34.478942 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.478849 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fb60fab-5cca-4668-b4c2-a469e4653f59-sys\") pod \"perf-node-gather-daemonset-s6x7k\" (UID: \"4fb60fab-5cca-4668-b4c2-a469e4653f59\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:34.478942 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.478866 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fb60fab-5cca-4668-b4c2-a469e4653f59-proc\") pod \"perf-node-gather-daemonset-s6x7k\" (UID: \"4fb60fab-5cca-4668-b4c2-a469e4653f59\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:34.478942 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.478903 2548 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fb60fab-5cca-4668-b4c2-a469e4653f59-lib-modules\") pod \"perf-node-gather-daemonset-s6x7k\" (UID: \"4fb60fab-5cca-4668-b4c2-a469e4653f59\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:34.580132 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.580051 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpkqp\" (UniqueName: \"kubernetes.io/projected/4fb60fab-5cca-4668-b4c2-a469e4653f59-kube-api-access-vpkqp\") pod \"perf-node-gather-daemonset-s6x7k\" (UID: \"4fb60fab-5cca-4668-b4c2-a469e4653f59\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:34.580132 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.580112 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fb60fab-5cca-4668-b4c2-a469e4653f59-podres\") pod \"perf-node-gather-daemonset-s6x7k\" (UID: \"4fb60fab-5cca-4668-b4c2-a469e4653f59\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:34.580132 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.580129 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fb60fab-5cca-4668-b4c2-a469e4653f59-sys\") pod \"perf-node-gather-daemonset-s6x7k\" (UID: \"4fb60fab-5cca-4668-b4c2-a469e4653f59\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:34.580392 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.580146 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fb60fab-5cca-4668-b4c2-a469e4653f59-proc\") pod \"perf-node-gather-daemonset-s6x7k\" (UID: \"4fb60fab-5cca-4668-b4c2-a469e4653f59\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:34.580392 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.580187 2548 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fb60fab-5cca-4668-b4c2-a469e4653f59-lib-modules\") pod \"perf-node-gather-daemonset-s6x7k\" (UID: \"4fb60fab-5cca-4668-b4c2-a469e4653f59\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:34.580392 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.580220 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fb60fab-5cca-4668-b4c2-a469e4653f59-sys\") pod \"perf-node-gather-daemonset-s6x7k\" (UID: \"4fb60fab-5cca-4668-b4c2-a469e4653f59\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:34.580392 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.580234 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4fb60fab-5cca-4668-b4c2-a469e4653f59-podres\") pod \"perf-node-gather-daemonset-s6x7k\" (UID: \"4fb60fab-5cca-4668-b4c2-a469e4653f59\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:34.580392 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.580288 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4fb60fab-5cca-4668-b4c2-a469e4653f59-proc\") pod \"perf-node-gather-daemonset-s6x7k\" (UID: \"4fb60fab-5cca-4668-b4c2-a469e4653f59\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:34.580392 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.580367 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4fb60fab-5cca-4668-b4c2-a469e4653f59-lib-modules\") pod \"perf-node-gather-daemonset-s6x7k\" (UID: \"4fb60fab-5cca-4668-b4c2-a469e4653f59\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:34.588826 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.588801 2548 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpkqp\" (UniqueName: \"kubernetes.io/projected/4fb60fab-5cca-4668-b4c2-a469e4653f59-kube-api-access-vpkqp\") pod \"perf-node-gather-daemonset-s6x7k\" (UID: \"4fb60fab-5cca-4668-b4c2-a469e4653f59\") " pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:34.644275 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.644248 2548 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:34.774140 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.774112 2548 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k"] Jul 18 02:34:34.775594 ip-10-0-137-108 kubenswrapper[2548]: W0718 02:34:34.775566 2548 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4fb60fab_5cca_4668_b4c2_a469e4653f59.slice/crio-42bd46f68382e085d73d9596b0fdce06237f6e52447bc01f915f96dca300399e WatchSource:0}: Error finding container 42bd46f68382e085d73d9596b0fdce06237f6e52447bc01f915f96dca300399e: Status 404 returned error can't find the container with id 42bd46f68382e085d73d9596b0fdce06237f6e52447bc01f915f96dca300399e Jul 18 02:34:34.949291 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.949258 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" event={"ID":"4fb60fab-5cca-4668-b4c2-a469e4653f59","Type":"ContainerStarted","Data":"f9dbeb5823cb2e2172ef9a97514f03692b3497d2f06eee69a7bc6893b7db9461"} Jul 18 02:34:34.949844 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.949771 2548 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" event={"ID":"4fb60fab-5cca-4668-b4c2-a469e4653f59","Type":"ContainerStarted","Data":"42bd46f68382e085d73d9596b0fdce06237f6e52447bc01f915f96dca300399e"} Jul 18 02:34:34.949972 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:34.949894 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:35.205764 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:35.205729 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/2.log" Jul 18 02:34:35.210710 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:35.210688 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-dhpvd_7d9e6ac4-e8b2-40ef-88de-f70735119ce5/console-operator/3.log" Jul 18 02:34:36.362110 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:36.362082 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-qnh5s_faf8aa57-0713-4c0e-8162-ed4ccb64e765/volume-data-source-validator/0.log" Jul 18 02:34:37.415923 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:37.415896 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zl6kk_5e4da607-7ffa-41ce-a806-b53359365b9a/dns/0.log" Jul 18 02:34:37.436692 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:37.436668 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zl6kk_5e4da607-7ffa-41ce-a806-b53359365b9a/kube-rbac-proxy/0.log" Jul 18 02:34:37.482108 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:37.482082 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bwx4x_2f1332dd-5e85-4bdf-b87a-ae1c5f147c3b/dns-node-resolver/0.log" Jul 18 02:34:38.052421 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:38.052381 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5n94l_2ded5998-7e45-4049-9302-bab9411039ba/node-ca/0.log" Jul 18 02:34:39.133739 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:39.133712 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c65d4b6f-g5q5h_56701e43-78f7-4acb-8f31-b3f75fc147af/kube-auth-proxy/0.log" Jul 18 02:34:39.184132 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:39.184105 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-62vpz_aeddc1c9-3b8b-4f5b-91a8-ac2655950649/istio-proxy/0.log" Jul 18 02:34:39.214640 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:39.214601 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-62vpz_aeddc1c9-3b8b-4f5b-91a8-ac2655950649/istio-proxy/1.log" Jul 18 02:34:39.286587 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:39.286562 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-v8tzt_6da9d422-36df-404f-a158-d3de9a1b6a6a/payload-processing/0.log" Jul 18 02:34:39.307052 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:39.307032 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-84775f69dd-wq2kv_27d13595-ae03-4459-aa8c-546b1c4500b0/router/0.log" Jul 18 02:34:39.973050 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:39.973023 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r222c_9b463fea-e348-40d3-8024-d09e8af88bd6/serve-healthcheck-canary/0.log" Jul 18 02:34:40.449683 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:40.449658 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-8zxk6_df23c9ba-7c94-4295-81a3-7452fb9ae1a9/insights-operator/0.log" Jul 18 02:34:40.450259 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:40.450241 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-8zxk6_df23c9ba-7c94-4295-81a3-7452fb9ae1a9/insights-operator/1.log" Jul 18 02:34:40.605374 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:40.605350 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xmx42_130558d4-120e-43cb-a644-e55e3321629b/kube-rbac-proxy/0.log" Jul 18 02:34:40.626431 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:40.626409 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xmx42_130558d4-120e-43cb-a644-e55e3321629b/exporter/0.log" Jul 18 02:34:40.646756 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:40.646731 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xmx42_130558d4-120e-43cb-a644-e55e3321629b/extractor/0.log" Jul 18 02:34:40.964058 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:40.964035 2548 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" Jul 18 02:34:40.986853 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:40.986811 2548 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-28rtf/perf-node-gather-daemonset-s6x7k" podStartSLOduration=6.986798107 podStartE2EDuration="6.986798107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:34:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:34:34.965725537 +0000 UTC m=+2783.818468880" watchObservedRunningTime="2026-07-18 02:34:40.986798107 +0000 UTC m=+2789.839541444" Jul 18 02:34:42.715815 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:42.715766 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-b8nbn_8a20675e-03ef-44a5-8663-ead09b5e6774/manager/0.log" Jul 18 02:34:42.890240 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:42.890212 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-pvq8t_c0e8558a-a0ba-4d9c-a4bc-4cab55db2a77/manager/0.log" Jul 18 02:34:43.003964 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:43.003886 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-75667cc8f5-q7hjf_1fa4f31e-6ff9-4085-b65d-f8c04aa7d17a/manager/0.log" Jul 18 02:34:43.030080 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:43.030060 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-bk8lx_bca4080e-2f9e-453e-8d6b-6179b8665cfd/server/0.log" Jul 18 02:34:43.102809 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:43.102772 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-g57qp_953b02cb-cd60-4aa0-b742-e5035254d741/manager/0.log" Jul 18 02:34:43.179368 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:43.179340 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c588c4c7-6wk76_6a9da053-d6e0-4e23-bbf0-1765e1ded023/manager/0.log" Jul 18 02:34:43.195713 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:43.195694 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c588c4c7-6wk76_6a9da053-d6e0-4e23-bbf0-1765e1ded023/copy-manifests/0.log" Jul 18 02:34:45.585211 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:45.585183 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c65d4b6f-g5q5h_56701e43-78f7-4acb-8f31-b3f75fc147af/kube-auth-proxy/0.log" Jul 18 02:34:45.620035 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:45.619955 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-62vpz_aeddc1c9-3b8b-4f5b-91a8-ac2655950649/istio-proxy/0.log" Jul 18 02:34:45.649698 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:45.649676 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-62vpz_aeddc1c9-3b8b-4f5b-91a8-ac2655950649/istio-proxy/1.log" Jul 18 02:34:45.710501 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:45.710479 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-v8tzt_6da9d422-36df-404f-a158-d3de9a1b6a6a/payload-processing/0.log" Jul 18 02:34:45.721844 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:45.721820 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-84775f69dd-wq2kv_27d13595-ae03-4459-aa8c-546b1c4500b0/router/0.log" Jul 18 02:34:47.004652 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:47.004623 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c65d4b6f-g5q5h_56701e43-78f7-4acb-8f31-b3f75fc147af/kube-auth-proxy/0.log" Jul 18 02:34:47.046887 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:47.046859 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-62vpz_aeddc1c9-3b8b-4f5b-91a8-ac2655950649/istio-proxy/0.log" Jul 18 02:34:47.076252 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:47.076228 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-62vpz_aeddc1c9-3b8b-4f5b-91a8-ac2655950649/istio-proxy/1.log" Jul 18 02:34:47.130713 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:47.130689 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-v8tzt_6da9d422-36df-404f-a158-d3de9a1b6a6a/payload-processing/0.log" Jul 18 02:34:47.146403 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:47.146385 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-84775f69dd-wq2kv_27d13595-ae03-4459-aa8c-546b1c4500b0/router/0.log" Jul 18 02:34:50.128480 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:50.128454 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-64rzk_2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1/kube-storage-version-migrator-operator/1.log" Jul 18 02:34:50.129272 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:50.129255 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-64rzk_2dee6aa6-deaa-4a24-bc80-93fbba3bf9d1/kube-storage-version-migrator-operator/0.log" Jul 18 02:34:51.094843 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:51.094810 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-98wvv_33094a8a-fe46-41a7-ba7a-cdf29c47e2d7/kube-multus/0.log" Jul 18 02:34:51.406958 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:51.406932 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvmsl_a8061f1d-195f-4a14-a518-5b0b0275471e/kube-multus-additional-cni-plugins/0.log" Jul 18 02:34:51.425910 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:51.425887 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvmsl_a8061f1d-195f-4a14-a518-5b0b0275471e/egress-router-binary-copy/0.log" Jul 18 02:34:51.443892 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:51.443874 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvmsl_a8061f1d-195f-4a14-a518-5b0b0275471e/cni-plugins/0.log" Jul 18 02:34:51.463470 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:51.463447 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvmsl_a8061f1d-195f-4a14-a518-5b0b0275471e/bond-cni-plugin/0.log" Jul 18 02:34:51.482275 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:51.482253 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvmsl_a8061f1d-195f-4a14-a518-5b0b0275471e/routeoverride-cni/0.log" Jul 18 02:34:51.500141 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:51.500123 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvmsl_a8061f1d-195f-4a14-a518-5b0b0275471e/whereabouts-cni-bincopy/0.log" Jul 18 02:34:51.518656 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:51.518637 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvmsl_a8061f1d-195f-4a14-a518-5b0b0275471e/whereabouts-cni/0.log" Jul 18 02:34:51.672105 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:51.672034 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7d9w7_fea0f11b-353d-4fe1-86bb-08286c333fd3/network-metrics-daemon/0.log" Jul 18 02:34:51.688490 ip-10-0-137-108 kubenswrapper[2548]: I0718 02:34:51.688472 2548 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-7d9w7_fea0f11b-353d-4fe1-86bb-08286c333fd3/kube-rbac-proxy/0.log"